summaryrefslogtreecommitdiff
path: root/progress.c
AgeCommit message (Collapse)AuthorFilesLines
2019-04-25Merge branch 'jk/xmalloc'Libravatar Junio C Hamano1-13/+5
The code is updated to check the result of memory allocation before it is used in more places, by using xmalloc and/or xcalloc calls. * jk/xmalloc: progress: use xmalloc/xcalloc xdiff: use xmalloc/xrealloc xdiff: use git-compat-util test-prio-queue: use xmalloc
2019-04-25Merge branch 'sg/overlong-progress-fix'Libravatar Junio C Hamano1-21/+53
Updating the display with progress message has been cleaned up to deal better with overlong messages. * sg/overlong-progress-fix: progress: break too long progress bar lines progress: clear previous progress update dynamically progress: assemble percentage and counters in a strbuf before printing progress: make display_progress() return void
2019-04-15progress: break too long progress bar linesLibravatar SZEDER Gábor1-3/+24
Some of the recently added progress indicators have quite long titles, which might be even longer when translated to some languages, and when they are shown while operating on bigger repositories, then the progress bar grows longer than the default 80 column terminal width. When the progress bar exceeds the width of the terminal it gets line-wrapped, and after that the CR at the end doesn't return to the beginning of the progress bar, but to the first column of its last line. Consequently, the first line of the previously shown progress bar is not overwritten by the next, and we end up with a bunch of truncated progress bar lines scrolling past: $ LANG=es_ES.UTF-8 git commit-graph write Encontrando commits para commit graph entre los objetos empaquetados: 2% (1599 Encontrando commits para commit graph entre los objetos empaquetados: 3% (1975 Encontrando commits para commit graph entre los objetos empaquetados: 4% (2633 Encontrando commits para commit graph entre los objetos empaquetados: 5% (3292 [...] Prevent this by breaking progress bars after the title once they exceed the width of the terminal, so the counter and optional percentage and throughput, i.e. all changing parts, are on the last line. Subsequent updates will from then on only refresh the changing parts, but not the title, and it will look like this: $ LANG=es_ES.UTF-8 ~/src/git/git commit-graph write Encontrando commits para commit graph entre los objetos empaquetados: 100% (6584502/6584502), listo. Calculando números de generación de commit graph: 100% (824705/824705), listo. Escribiendo commit graph en 4 pasos: 100% (3298820/3298820), listo. Note that the number of columns in the terminal is cached by term_columns(), so this might not kick in when it should when a terminal window is resized while the operation is running. Furthermore, this change won't help if the terminal is so narrow that the counters don't fit on one line, but I would put this in the "If it hurts, don't do it" box. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15progress: clear previous progress update dynamicallyLibravatar SZEDER Gábor1-4/+7
When the progress bar includes throughput, its length can shorten as the unit of display changes from KiB to MiB. To cover up remnants of the previous progress bar when such a change of units happens we always print three spaces at the end of the progress bar. Alas, covering only three characters is not quite enough: when both the total and the throughput happen to change units from KiB to MiB in the same update, then the progress bar's length is shortened by four characters (or maybe even more!): Receiving objects: 25% (2901/11603), 772.01 KiB | 733.00 KiB/s Receiving objects: 27% (3133/11603), 1.43 MiB | 1.16 MiB/s s and a stray 's' is left behind. So instead of hard-coding the three characters to cover, let's compare the length of the current progress bar with the previous one, and cover up as many characters as needed. Sure, it would be much simpler to just print more spaces at the end of the progress bar, but this approach is more future-proof, and it won't print extra spaces when none are needed, notably when the progress bar doesn't show throughput and thus never shrinks. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12progress: use xmalloc/xcallocLibravatar Jeff King1-13/+5
Since the early days of Git, the progress code allocates its struct with a bare malloc(), not xmalloc(). If the allocation fails, we just avoid showing progress at all. While perhaps a noble goal not to fail the whole operation because of optional progress, in practice: 1. Any failure to allocate a few dozen bytes here means critical path allocations are likely to fail, too. 2. These days we use a strbuf for throughput progress (and there's a patch under discussion to do the same for non-throughput cases, too). And that uses xmalloc() under the hood, which means we'd still die on some allocation failures. Let's switch to xmalloc(). That makes us consistent with the rest of Git and makes it easier to audit for other (less careful) bare mallocs. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-05progress: assemble percentage and counters in a strbuf before printingLibravatar SZEDER Gábor1-12/+23
The following patches in this series want to handle the progress bar's title and changing parts (i.e. the counter and the optional percentage and throughput combined) differently, and need to know the length of the changing parts of the previously displayed progress bar. To prepare for those changes assemble the changing parts in a separate strbuf kept in 'struct progress' before printing. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-05progress: make display_progress() return voidLibravatar SZEDER Gábor1-8/+5
Ever since the progress infrastructure was introduced in 96a02f8f6d (common progress display support, 2007-04-18), display_progress() has returned an int, telling callers whether it updated the progress bar or not. However, this is: - useless, because over the last dozen years there has never been a single caller that cared about that return value. - not quite true, because it doesn't print a progress bar when running in the background, yet it returns 1; see 85cb8906f0 (progress: no progress in background, 2015-04-13). The related display_throughput() function returned void already upon its introduction in cf84d51c43 (add throughput to progress display, 2007-10-30). Let's make display_progress() return void, too. While doing so several return statements in display() become unnecessary, remove them. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-22progress: add sparse mode to force 100% complete messageLibravatar Jeff Hostetler1-3/+35
Add new start_sparse_progress() and start_delayed_sparse_progress() constructors and "sparse" flag to struct progress. Teach stop_progress() to force a 100% complete progress message before printing the final "done" message when "sparse" is set. Calling display_progress() for every item in a large set can be expensive. If callers try to filter this for performance reasons, such as emitting every k-th item, progress would not reach 100% unless they made a final call to display_progress() with the item count before calling stop_progress(). Now this is automatic when "sparse" is set. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-19Merge branch 'en/rename-progress'Libravatar Junio C Hamano1-14/+15
Historically, the diff machinery for rename detection had a hardcoded limit of 32k paths; this is being lifted to allow users trade cycles with a (possibly) easier to read result. * en/rename-progress: diffcore-rename: make diff-tree -l0 mean -l<large> sequencer: show rename progress during cherry picks diff: remove silent clamp of renameLimit progress: fix progress meters when dealing with lots of work sequencer: warn when internal merge may be suboptimal due to renameLimit
2017-12-04progress: drop delay-threshold codeLibravatar Lars Schneider1-19/+5
Since 180a9f2268 (provide a facility for "delayed" progress reporting, 2007-04-20), the progress code has allowed callers to skip showing progress if they have reached a percentage-threshold of the total work before the delay period passes. But since 8aade107dd (progress: simplify "delayed" progress API, 2017-08-19), that parameter is not available to outside callers (we always passed zero after that commit, though that was corrected in the previous commit to "100%"). Let's drop the threshold code, which never triggers in any meaningful way. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-04progress: set default delay threshold to 100%, not 0%Libravatar Jeff King1-1/+1
Commit 8aade107dd (progress: simplify "delayed" progress API, 2017-08-19) dropped the parameter by which callers could say "show my progress only if I haven't passed M% progress after N seconds". The intent was to just show nothing for 2 seconds, and then always progress after that. But we flipped the logic in the wrapper: it sets M=0, meaning that we'd almost _never_ show progress after 2 seconds, since we'd generally have made some progress. This should have been 100%, not 0%. We were fooled by existing calls like: start_progress_delay("foo", 0, 0, 2); which behaved this way. The trick is that the first "0" there is "how many items total", and there zero means "we don't know". And without knowing that, we cannot compute a completed percent at all, and we ignored the threshold parameter entirely! Modeling our wrapper after that broke callers which pass a non-zero value for "total". We can switch to the intended behavior by using "100" in the wrapper call. Reported-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-15progress: fix progress meters when dealing with lots of workLibravatar Elijah Newren1-14/+15
The possibility of setting merge.renameLimit beyond 2^16 raises the possibility that the values passed to progress can exceed 2^32. Use uint64_t, because it "ought to be enough for anybody". :-) Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-19progress: simplify "delayed" progress APILibravatar Junio C Hamano1-5/+10
We used to expose the full power of the delayed progress API to the callers, so that they can specify, not just the message to show and expected total amount of work that is used to compute the percentage of work performed so far, the percent-threshold parameter P and the delay-seconds parameter N. The progress meter starts to show at N seconds into the operation only if we have not yet completed P per-cent of the total work. Most callers used either (0%, 2s) or (50%, 1s) as (P, N), but there are oddballs that chose more random-looking values like 95%. For a smoother workload, (50%, 1s) would allow us to start showing the progress meter earlier than (0%, 2s), while keeping the chance of not showing progress meter for long running operation the same as the latter. For a task that would take 2s or more to complete, it is likely that less than half of it would complete within the first second, if the workload is smooth. But for a spiky workload whose earlier part is easier, such a setting is likely to fail to show the progress meter entirely and (0%, 2s) is more appropriate. But that is merely a theory. Realistically, it is of dubious value to ask each codepath to carefully consider smoothness of their workload and specify their own setting by passing two extra parameters. Let's simplify the API by dropping both parameters and have everybody use (0%, 2s). Oh, by the way, the percent-threshold parameter and the structure member were consistently misspelled, which also is now fixed ;-) Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-09progress: show overall rate in last updateLibravatar René Scharfe1-2/+6
The values in struct throughput are only updated every 0.5 seconds. If we're all done before that time span then the final update will show a rate of 0 bytes/s, which is misleading if some bytes had been handled. Remember the start time and show the total throughput instead. And avoid division by zero by enforcing a minimum time span value of 1 (unit: 1/1024th of a second). That makes the resulting rate an underestimation, but it's closer to the actual value than the currently shown 0 bytes/s. Reported-by: 積丹尼 Dan Jacobson <jidanni@jidanni.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-16stop_progress_msg: convert xsnprintf to xstrfmtLibravatar Maxim Moseychuk1-7/+4
Simplify code by replacing buffer allocation with a call to xstrfmt(). Signed-off-by: Maxim Moseychuk <franchesko.salias.hudro.pedros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22use xmallocz to avoid size arithmeticLibravatar Jeff King1-1/+1
We frequently allocate strings as xmalloc(len + 1), where the extra 1 is for the NUL terminator. This can be done more simply with xmallocz, which also checks for integer overflow. There's no case where switching xmalloc(n+1) to xmallocz(n) is wrong; the result is the same length, and malloc made no guarantees about what was in the buffer anyway. But in some cases, we can stop manually placing NUL at the end of the allocated buffer. But that's only safe if it's clear that the contents will always fill the buffer. In each case where this patch does so, I manually examined the control flow, and I tried to err on the side of caution. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25stop_progress_msg: convert sprintf to xsnprintfLibravatar Jeff King1-1/+1
The usual arguments for using xsnprintf over sprintf apply, but this case is a little tricky. We print to a fixed-size buffer if we have room, and otherwise to an allocated buffer. So there should be no overflow here, but it is still good to communicate our intention, as well as to check our earlier math for how much space the string will need. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25progress: store throughput display in a strbufLibravatar Jeff King1-10/+8
Coverity noticed that we strncpy() into a fixed-size buffer without making sure that it actually ended up NUL-terminated. This is unlikely to be a bug in practice, since throughput strings rarely hit 32 characters, but it would be nice to clean it up. The most obvious way to do so is to add a NUL-terminator. But instead, this patch switches the fixed-size buffer out for a strbuf. At first glance this seems much less efficient, until we realize that filling in the fixed-size buffer is done by writing into a strbuf and copying the result! By writing straight to the buffer, we actually end up more efficient: 1. We avoid an extra copy of the bytes. 2. Rather than malloc/free each time progress is shown, we can strbuf_reset and use the same buffer each time. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-19progress: treat "no terminal" as being in the foregroundLibravatar Jeff King1-1/+2
progress: treat "no terminal" as being in the foreground Commit 85cb890 (progress: no progress in background, 2015-04-13) avoids sending progress from background processes by checking that the process group id of the current process is the same as that of the controlling terminal. If we don't have a terminal, however, this check never succeeds, and we print no progress at all (until the final "done" message). This can be seen when cloning a large repository; instead of getting progress updates for "counting objects", it will appear to hang then print the final count. We can fix this by treating an error return from tcgetpgrp() as a signal to show the progress. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-15progress: no progress in backgroundLibravatar Luke Mewburn1-6/+16
Disable the display of the progress if stderr is not the current foreground process. Still display the final result when done. Signed-off-by: Luke Mewburn <luke@mewburn.net> Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-13progress: simplify performance measurement by using getnanotime()Libravatar Karsten Blees1-35/+36
Calculating duration from a single uint64_t is simpler than from a struct timeval. Change throughput measurement from gettimeofday() to getnanotime(). Also calculate misec only if needed, and change integer division to integer multiplication + shift, which should be slightly faster. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-02-24i18n: mark all progress lines for translationLibravatar Nguyễn Thái Ngọc Duy1-1/+2
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-10strbuf: create strbuf_humanise_bytes() to show byte sizesLibravatar Antoine Pelisse1-28/+15
Humanization of downloaded size is done in the same function as text formatting in 'process.c'. The code cannot be reused easily elsewhere. Separate text formatting from size simplification and make the function public in strbuf so that it can easily be used by other callers. We now can use strbuf_humanise_bytes() for both downloaded size and download speed calculation. One of the drawbacks is that speed will now look like this when download is stalled: "0 bytes/s" instead of "0 KiB/s". Signed-off-by: Antoine Pelisse <apelisse@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-10-14change throughput display units with fast linksLibravatar Nicolas Pitre1-1/+7
Switch to MiB/s when the connection is fast enough (i.e. on a LAN). Signed-off-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-09-14Nicolas Pitre has a new email addressLibravatar Nicolas Pitre1-1/+1
Due to problems at cam.org, my nico@cam.org email address is no longer valid. From now on, nico@fluxnic.net should be used instead. Signed-off-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-04-25progress bar: round to the nearest instead of truncating downLibravatar Nicolas Pitre1-4/+4
Often the throughput output is requested when the data read so far is one smaller than multiple of 1024; because 1023/1024 is ~0.999, it often shows up as 0.99 because the code currently truncates. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08progress.c: avoid use of dynamic-sized arrayLibravatar Boyd Lynn Gerber1-3/+8
Dynamically sized arrays are gcc and C99 construct. Using them hurts portability to older compilers, although using them is nice in this case it is not desirable. This patch removes the only use of the construct in stop_progress_msg(); the function is about writing out a single line of a message, and the existing callers of this function feed messages of only bounded size anyway, so use of dynamic array is simply overkill. Signed-off-by: Boyd Lynn Gerber <gerberb@zenez.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-20Flush progress message buffer in display().Libravatar Johannes Sixt1-0/+3
This will make progress display from pack-objects (invoked via upload-pack) more responsive on platforms with an implementation of stdio whose stderr is line buffered. The standard error stream is defined to be merely "not fully buffered"; this is different from "unbuffered". If the implementation of the stdio library chooses to make it line buffered, progress reports that end with CR but do not contain LF will accumulate in the stdio buffer before written out. A fflush() after each progress display gives a nice continuous progress. Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-08nicer display of thin pack completionLibravatar Nicolas Pitre1-6/+13
In the same spirit of prettifying Git's output display for mere mortals, here's a simple extension to the progress API allowing for a final message to be provided when terminating a progress line, and use it for the display of the number of objects needed to complete a thin pack, saving yet one more line of screen display. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-07make display of total transferred fully accurateLibravatar Nicolas Pitre1-25/+39
The minimum delay of 1/2 sec between successive throughput updates might not have been elapsed when display_throughput() is called for the last time, potentially making the display of total transferred bytes not right when progress is said to be done. Let's force an update of the throughput display as well when the progress is complete. As a side effect, the total transferred will always be displayed even if the actual transfer rate doesn't have time to kickin. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-05make display of total transferred more accurateLibravatar Nicolas Pitre1-24/+22
The throughput display needs a delay period before accounting and displaying anything. Yet it might be called after some amount of data has already been transferred. The display of total data is therefore accounted late and therefore smaller than the reality. Let's call display_throughput() with an absolute amount of transferred data instead of a relative number, and let the throughput code find the relative amount of data by itself as needed. This way the displayed total is always exact. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-01Show total transferred as part of throughput progressLibravatar Nicolas Pitre1-3/+26
Right now it is infeasible to offer to the user a reasonable concept of when a clone will be complete as we aren't able to come up with the final pack size until after we have actually transferred the entire thing to the client. However in many cases users can work with a rough rule-of-thumb; for example it is somewhat well known that git.git is about 16 MiB today and that linux-2.6.git is over 120 MiB. We now show the total amount of data we have transferred over the network as part of the throughput meter, organizing it in "human friendly" terms like `ls -h` would do. Users can glance at this, see that the total transferred size is about 3 MiB, see the throughput of X KiB/sec, and determine a reasonable figure of about when the clone will be complete, assuming they know the rough size of the source repository or are able to obtain it. This is also a helpful indicator that there is progress being made even if we stall on a very large object. The thoughput meter may remain relatively constant and the percentage complete and object count won't be changing, but the total transferred will be increasing as additional data is received for this object. [from an initial proposal from Shawn O. Pearce] Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-01make sure throughput display gets updated even if progress doesn't moveLibravatar Nicolas Pitre1-0/+3
Currently the progress/throughput display update happens only through display_progress(). If the progress based on object count remains unchanged because a large object is being received, the latest throughput won't be displayed. The display update should occur through display_throughput() as well. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30add some copyright notice to the progress display codeLibravatar Nicolas Pitre1-0/+10
Some self patting on the back to keep my ego alive. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30add throughput to progress displayLibravatar Nicolas Pitre1-4/+76
This adds the ability for the progress code to also display transfer throughput when that makes sense. The math was inspired by commit c548cf4ee0737a321ffe94f6a97c65baf87281be from Linus. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30make struct progress an opaque typeLibravatar Nicolas Pitre1-6/+27
This allows for better management of progress "object" existence, as well as making the progress display implementation more independent from its callers. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-17more compact progress displayLibravatar Nicolas Pitre1-28/+25
Each progress can be on a single line instead of two. [sp: Changed "Checking files out" to "Checking out files" at Johannes Sixt's suggestion as it better explains the action that is taking place] Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-05-23Fix the progress code to output LF only when it is really neededLibravatar Alex Riesen1-1/+5
Signed-off-by: Alex Riesen <raa.lkml@gmail.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-04-22provide a facility for "delayed" progress reportingLibravatar Nicolas Pitre1-0/+34
This allows for progress to be displayed only if the progress has not reached a specified percentage treshold within a given delay in seconds. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-04-22make progress "title" part of the common progress interfaceLibravatar Nicolas Pitre1-4/+8
If the progress bar ends up in a box, better provide a title for it too. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-04-22common progress display supportLibravatar Nicolas Pitre1-0/+68
Instead of having this code duplicated in multiple places, let's have a common interface for progress display. If someday someone wishes to display a cheezy progress bar instead then only one file will have to be changed. Note: I left merge-recursive.c out since it has a strange notion of progress as it apparently increase the expected total number as it goes. Someone with more intimate knowledge of what that is supposed to mean might look at converting it to the common progress interface. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>