Fix CPU owning for curl downloads while on slow internet connections when Content-Length header is not present.

App Engine Blobstore does not send Content-Type, which means targetsize is always 0 and therefore the conditional that calculates and reports download progress is never entered, and the existing sleep location is never hit. On slow internet connections, this means that CPU spikes as proc.poll() is being called far more than curl is downloading.

Fix this by moving time.sleep(0.1) into the main curl while loop, where it'll execute on every loop iteration. That means we have a small sleep even while headers are downloading/etc, but that's not a big deal given it's a 1/10th of a second.
This commit is contained in:
Justin McWilliams
2011-09-15 18:39:49 -04:00
parent 25a9315cba
commit 79cec41552
+3 -2
View File
@@ -2432,7 +2432,8 @@ def curl(url, destinationpath, onlyifnewer=False, etag=None, resume=False,
# percent changed; update display
downloadedpercent = percent
munkicommon.display_percent_done(downloadedpercent, 100)
time.sleep(0.1)
time.sleep(0.1)
if (proc.poll() != None):
break
@@ -3068,7 +3069,7 @@ def check(client_id='', localmanifestpath=None):
def discardTimeZoneFromDate(the_date):
"""Input: NSDate object
Output: NSDate object with same date and time as the UTC.
In Los Angeles (PDT), '2011-06-20T12:00:00Z' becomes
In Los Angeles (PDT), '2011-06-20T12:00:00Z' becomes
'2011-06-20 12:00:00 -0700'.
In New York (EDT), it becomes '2011-06-20 12:00:00 -0400'.
"""