RE: Issue : Writing commits into the git repository takes longer than expected

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi there! 

Is there any update on this?

Thanks

-----Original Message-----
From: Arpit Gupta 
Sent: Wednesday, June 12, 2024 5:15 PM
To: rsbecker@xxxxxxxxxxxxx; git@xxxxxxxxxxxxxxx
Cc: Anuradha Patial <anpatial@xxxxxxxxx>; Madhurima Pandey <madhupandey@xxxxxxxxx>
Subject: RE: Issue : Writing commits into the git repository takes longer than expected

>> The XML files that are being written as content are multi-line. There are 2 tags present in the file and each tag are on their own line (one tag being the child of the other). The file size isn't large. It is hardly 2-3kb. Below is the sample structure of the XML file being added as a part of content:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <ServiceName type=ServiceType> <property>PropertyValue</property>
</ServiceName>

>> There are no virus scans running in the repository. Also, the git LFS isn't involved in this scenario.
>> There might be a case as when the commit time starts increasing (initially from 4-5s to 30s to 1min to 6-7min) and during that time another commit call also starts as there is a scheduler of 5 minutes which triggers this action. But this will only cause a certain amount of delay and it shouldn't be the factor to increase the CPU Utilization.
Also, the machine memory size is 32GB. 

The commit time starts increasing from 4-5s and goes up to 6-7mins, what could be the trigger for the commit to increase from 4-5s to 1min and so on in this scenario since before that there can't be any parallel commits ongoing onto the repository? Also, as I mentioned before, this issue is totally inconsistent.
Let me know in case any other information is required.

Thanks & Regards,
Arpit

-----Original Message-----
From: rsbecker@xxxxxxxxxxxxx <rsbecker@xxxxxxxxxxxxx>
Sent: Monday, June 10, 2024 6:29 PM
To: Arpit Gupta <argupta@xxxxxxxxx>; git@xxxxxxxxxxxxxxx
Subject: RE: Issue : Writing commits into the git repository takes longer than expected

[You don't often get email from rsbecker@xxxxxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

On Monday, June 10, 2024 6:17 AM, Arpit Gupta wrote:
>We are maintaining the different versions of data in git repository 
>using
jgit maven
>library. So, a commit is done on the repository containing properties 
>such
as author
>name, date and time, action, and the file path.
>The file path refers the xml file which contains the action performed 
>and
is stored
>inside the repository.
>
>We have a job running every 5 minutes that commits the information onto 
>the repository and the XML file content is over-written every time.
>Usually,
the commits
>and writing of XML file takes around 4-5 seconds but sometimes the time
while
>committing as well as writing the data increases which also increase 
>the
overall CPU
>utilization of the machine. This behavior is inconsistent with respect 
>to
the process
>and occurs randomly but during this behavior, there is a time when the 
>CPU utilization becomes high that all other running processes hangs up 
>which
demands
>the restart of the server.
>
>Could you please suggest which areas should we look for while 
>identifying
the cause
>of this issue? Also, does frequent commit of the content onto 
>repository
can trigger
>this issue?
>In your view, what might be the trigger of this issue and how we can
proceed to
>resolve it?

Are your XML files single line file or is each tag on its own line? Changes to single-line XML files can cause complete rewrites. If the file is large enough, this can cause performance issues.

Do you have virus scans running on your repository? These can also cause issues. Some scanners are more friendly to developers than others. Also, is this an NFS drive? Is Git LFS involved?

If you have two commits to the same repo happening at once, this can also cause one commit to be delayed waiting on the lock file. More info is needed to comment further.

--Randall






[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux