summaryrefslogtreecommitdiff
path: root/Makefile
AgeCommit message (Collapse)AuthorFilesLines
2022-03-16Merge branch 'jc/stash-drop'Libravatar Junio C Hamano1-0/+1
"git stash drop" is reimplemented as an internal call to reflog_delete() function, instead of invoking "git reflog delete" via run_command() API. * jc/stash-drop: stash: call reflog_delete() in reflog.c reflog: libify delete reflog function and helpers stash: add tests to ensure reflog --rewrite --updatref behavior
2022-03-03Makefiles: add and use wildcard "mkdir -p" templateLibravatar Ævar Arnfjörð Bjarmason1-5/+7
Add a template to do the "mkdir -p" of $(@D) (the parent dir of $@) for us, and use it for the "make lint-docs" targets I added in 8650c6298c1 (doc lint: make "lint-docs" non-.PHONY, 2021-10-15). As seen in 4c64fb5aad9 (Documentation/Makefile: fix lint-docs mkdir dependency, 2021-10-26) maintaining these manual lists of parent directory dependencies is fragile, in addition to being obviously verbose. I used this pattern at the time because I couldn't find another method than "order-only" prerequisites to avoid doing a "mkdir -p $(@D)" for every file being created, which as noted in [1] would be significantly slower. But as it turns out we can use this neat trick of only doing a "mkdir -p" if the $(wildcard) macro tells us the path doesn't exist. A re-run of a performance test similar to that noted downthread of [1] in [2] shows that this is faster, in addition to being less verbose and more reliable (this uses my "git-hyperfine" thin wrapper for "hyperfine"[3]): $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -C Documentation lint-docs' -p 'rm -rf Documentation/.build' 'make -C Documentation -j1 lint-docs' Benchmark 1: make -C Documentation -j1 lint-docs' in 'HEAD~1 Time (mean ± σ): 2.914 s ± 0.062 s [User: 2.449 s, System: 0.489 s] Range (min … max): 2.834 s … 3.020 s 10 runs Benchmark 2: make -C Documentation -j1 lint-docs' in 'HEAD~0 Time (mean ± σ): 2.315 s ± 0.062 s [User: 1.950 s, System: 0.386 s] Range (min … max): 2.229 s … 2.397 s 10 runs Summary 'make -C Documentation -j1 lint-docs' in 'HEAD~0' ran 1.26 ± 0.04 times faster than 'make -C Documentation -j1 lint-docs' in 'HEAD~1' So let's use that pattern both for the "lint-docs" target, and a few miscellaneous other targets. This method of creating parent directories is explicitly racy in that we don't know if we're going to say always create a "foo" followed by a "foo/bar" under parallelism, or skip the "foo" because we created "foo/bar" first. In this case it doesn't matter for anything except that we aren't guaranteed to get the same number of rules firing when running make in parallel. 1. https://lore.kernel.org/git/211028.861r45y3pt.gmgdl@evledraar.gmail.com/ 2. https://lore.kernel.org/git/211028.86o879vvtp.gmgdl@evledraar.gmail.com/ 3. https://gitlab.com/avar/git-hyperfine/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03Makefile: add "$(QUIET)" boilerplate to shared.makLibravatar Ævar Arnfjörð Bjarmason1-33/+0
The $(QUIET) variables we define are largely duplicated between our various Makefiles, let's define them in the new "shared.mak" instead. Since we're not using the environment to pass these around we don't need to export the "QUIET_GEN" and "QUIET_BUILT_IN" variables anymore. The "QUIET_GEN" variable is used in "git-gui/Makefile" and "gitweb/Makefile", but they've got their own definition for those. The "QUIET_BUILT_IN" variable is only used in the top-level "Makefile". We still need to export the "V" variable. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03Makefile: move $(comma), $(empty) and $(space) to shared.makLibravatar Ævar Arnfjörð Bjarmason1-4/+0
Move these variables over to the shared.mak, we'll make use of them in a subsequent commit. Note that there's reason for these to be "simply expanded variables", i.e. to use ":=" assignments instead of lazily expanded "=" assignments. We could use "=", but let's leave this as-is for now for ease of review. See 425ca6710b2 (Makefile: allow combining UBSan with other sanitizers, 2017-07-15) for the commit that introduced these. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03Makefile: move ".SUFFIXES" rule to shared.makLibravatar Ævar Arnfjörð Bjarmason1-2/+0
This was added in 30248886ce8 (Makefile: disable default implicit rules, 2010-01-26), let's move it to the top of "shared.mak" so it'll apply to all our Makefiles. This doesn't benefit the main Makefile at all, since it already had the rule, but since we're including shared.mak in other Makefiles starts to benefit them. E.g. running the 'man" target is now faster: $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -C Documentation man' 'make -C Documentation -j1 man' Benchmark 1: make -C Documentation -j1 man' in 'HEAD~1 Time (mean ± σ): 121.7 ms ± 8.8 ms [User: 105.8 ms, System: 18.6 ms] Range (min … max): 112.8 ms … 148.4 ms 26 runs Benchmark 2: make -C Documentation -j1 man' in 'HEAD~0 Time (mean ± σ): 97.5 ms ± 8.0 ms [User: 80.1 ms, System: 20.1 ms] Range (min … max): 89.8 ms … 111.8 ms 32 runs Summary 'make -C Documentation -j1 man' in 'HEAD~0' ran 1.25 ± 0.14 times faster than 'make -C Documentation -j1 man' in 'HEAD~1' The reason for that can be seen when comparing that run with "--debug=a". Without this change making a target like "git-status.1" will cause "make" to consider not only "git-status.txt", but "git-status.txt.o", as well as numerous other implicit suffixes such as ".c", ".cc", ".cpp" etc. See [1] for a more detailed before/after example. So this is causing us to omit a bunch of work we didn't need to do. For making "git-status.1" the "--debug=a" output is reduced from ~140k lines to ~6k. 1. https://lore.kernel.org/git/220222.86bkyz875k.gmgdl@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)Libravatar Ævar Arnfjörð Bjarmason1-28/+26
Combine the definitions of $(FIND_SOURCE_FILES) and $(LIB_H) to speed up the Makefile, as these are the two main expensive $(shell) commands that we execute unconditionally. When see what was in $(FOUND_SOURCE_FILES) that wasn't in $(LIB_H) via the ad-hoc test of: $(error $(filter-out $(LIB_H),$(filter %.h,$(ALL_SOURCE_FILES)))) $(error $(filter-out $(ALL_SOURCE_FILES),$(filter %.h,$(LIB_H)))) We'll get, respectively: Makefile:850: *** t/helper/test-tool.h. Stop. Makefile:850: *** . Stop. I.e. we only had a discrepancy when it came to t/helper/test-tool.h. In terms of correctness this was broken before, but now works: $ make t/helper/test-tool.hco HDR t/helper/test-tool.h This speeds things up a lot: $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make NO_TCLTK=Y' 'make -j1 NO_TCLTK=Y' --warmup 10 -M 10 Benchmark 1: make -j1 NO_TCLTK=Y' in 'HEAD~1 Time (mean ± σ): 159.9 ms ± 6.8 ms [User: 137.2 ms, System: 28.0 ms] Range (min … max): 154.6 ms … 175.9 ms 10 runs Benchmark 2: make -j1 NO_TCLTK=Y' in 'HEAD~0 Time (mean ± σ): 100.0 ms ± 1.3 ms [User: 84.2 ms, System: 20.2 ms] Range (min … max): 98.8 ms … 102.8 ms 10 runs Summary 'make -j1 NO_TCLTK=Y' in 'HEAD~0' ran 1.60 ± 0.07 times faster than 'make -j1 NO_TCLTK=Y' in 'HEAD~1' Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to itLibravatar Ævar Arnfjörð Bjarmason1-10/+3
We have various behavior that's shared across our Makefiles, or that really should be (e.g. via defined templates). Let's create a top-level "shared.mak" to house those sorts of things, and start by adding the ".DELETE_ON_ERROR" flag to it. See my own 7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29) and db10fc6c09f (doc: simplify Makefile using .DELETE_ON_ERROR, 2021-05-21) for the addition and use of the ".DELETE_ON_ERROR" flag. I.e. this changes the behavior of existing rules in the altered Makefiles (except "Makefile" & "Documentation/Makefile"). I'm confident that this is safe having read the relevant rules in those Makfiles, and as the GNU make manual notes that it isn't the default behavior is out of an abundance of backwards compatibility caution. From edition 0.75 of its manual, covering GNU make 4.3: [Enabling '.DELETE_ON_ERROR' is] almost always what you want 'make' to do, but it is not historical practice; so for compatibility, you must explicitly request it. This doesn't introduce a bug by e.g. having this ".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles have no such scoping semantics. It does increase the danger that any Makefile without an explicit "The default target of this Makefile is..." snippet to define the default target as "all" could have its default rule changed if our new shared.mak ever defines a "real" rule. In subsequent commits we'll be careful not to do that, and such breakage would be obvious e.g. in the case of "make -C t". We might want to make that less fragile still (e.g. by using ".DEFAULT_GOAL" as noted in the preceding commit), but for now let's simply include "shared.mak" without adding that boilerplate to all the Makefiles that don't have it already. Most of those are already exposed to that potential caveat e.g. due to including "config.mak*". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-02reflog: libify delete reflog function and helpersLibravatar John Cai1-0/+1
Currently stash shells out to reflog in order to delete refs. In an effort to reduce how much we shell out to a subprocess, libify the functionality that stash needs into reflog.c. Add a reflog_delete function that is pretty much the logic in the while loop in builtin/reflog.c cmd_reflog_delete(). This is a function that builtin/reflog.c and builtin/stash.c can both call. Also move functions needed by reflog_delete and export them. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: John Cai <johncai86@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-16Merge branch 'ab/auto-detect-zlib-compress2'Libravatar Junio C Hamano1-8/+3
The build procedure has been taught to notice older version of zlib and enable our replacement uncompress2() automatically. * ab/auto-detect-zlib-compress2: compat: auto-detect if zlib has uncompress2()
2022-02-11Merge branch 'bc/csprng-mktemps'Libravatar Junio C Hamano1-0/+34
Pick a better random number generator and use it when we prepare temporary filenames. * bc/csprng-mktemps: wrapper: use a CSPRNG to generate random file names wrapper: add a helper to generate numbers from a CSPRNG
2022-02-09Merge branch 'ab/config-based-hooks-2'Libravatar Junio C Hamano1-0/+1
More "config-based hooks". * ab/config-based-hooks-2: run-command: remove old run_hook_{le,ve}() hook API receive-pack: convert push-to-checkout hook to hook.h read-cache: convert post-index-change to use hook.h commit: convert {pre-commit,prepare-commit-msg} hook to hook.h git-p4: use 'git hook' to run hooks send-email: use 'git hook run' for 'sendemail-validate' git hook run: add an --ignore-missing flag hooks: convert worktree 'post-checkout' hook to hook library hooks: convert non-worktree 'post-checkout' hook to hook library merge: convert post-merge to use hook.h am: convert applypatch-msg to use hook.h rebase: convert pre-rebase to use hook.h hook API: add a run_hooks_l() wrapper am: convert {pre,post}-applypatch to use hook.h gc: use hook library for pre-auto-gc hook hook API: add a run_hooks() wrapper hook: add 'run' subcommand
2022-01-26compat: auto-detect if zlib has uncompress2()Libravatar Ævar Arnfjörð Bjarmason1-8/+3
We have a copy of uncompress2() implementation in compat/ so that we can build with an older version of zlib that lack the function, and the build procedure selects if it is used via the NO_UNCOMPRESS2 $(MAKE) variable. This is yet another "annoying" knob the porters need to tweak on platforms that are not common enough to have the default set in the config.mak.uname file. Attempt to instead ask the system header <zlib.h> to decide if we need the compatibility implementation. This is a deviation from the way we have been handling the "compatiblity" features so far, and if it can be done cleanly enough, it could work as a model for features that need compatibility definition we discover in the future. With that goal in mind, avoid expedient but ugly hacks, like shoving the code that is conditionally compiled into an unrelated .c file, which may not work in future cases---instead, take an approach that uses a file that is independently compiled and stands on its own. Compile and link compat/zlib-uncompress2.c file unconditionally, but conditionally hide the implementation behind #if/#endif when zlib version is 1.2.9 or newer, and unconditionally archive the resulting object file in the libgit.a to be picked up by the linker. There are a few things to note in the shape of the code base after this change: - We no longer use NO_UNCOMPRESS2 knob; if the system header <zlib.h> claims a version that is more cent than the library actually is, this would break, but it is easy to add it back when we find such a system. - The object file compat/zlib-uncompress2.o is always compiled and archived in libgit.a, just like a few other compat/ object files already are. - The inclusion of <zlib.h> is done in <git-compat-util.h>; we used to do so from <cache.h> which includes <git-compat-util.h> as the first thing it does, so from the *.c codes, there is no practical change. - Until objects in libgit.a that is already used gains a reference to the function, the reftable code will be the only one that wants it, so libgit.a on the linker command line needs to appear once more at the end to satisify the mutual dependency. - Beat found a trick used by OpenSSL to avoid making the conditionally-compiled object truly empty (apparently because they had to deal with compilers that do not want to see an effectively empty input file). Our compat/zlib-uncompress2.c file borrows the same trick for portabilty. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: Beat Bolli <dev+git@drbeat.li> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-17wrapper: add a helper to generate numbers from a CSPRNGLibravatar brian m. carlson1-0/+34
There are many situations in which having access to a cryptographically secure pseudorandom number generator (CSPRNG) is helpful. In the future, we'll encounter one of these when dealing with temporary files. To make this possible, let's add a function which reads from a system CSPRNG and returns some bytes. We know that all systems will have such an interface. A CSPRNG is required for a secure TLS or SSH implementation and a Git implementation which provided neither would be of little practical use. In addition, POSIX is set to standardize getentropy(2) in the next version, so in the (potentially distant) future we can rely on that. For systems which lack one of the other interfaces, we provide the ability to use OpenSSL's CSPRNG. OpenSSL is highly portable and functions on practically every known OS, and we know it will have access to some source of cryptographically secure randomness. We also provide support for the arc4random in libbsd for folks who would prefer to use that. Because this is a security sensitive interface, we take some precautions. We either succeed by filling the buffer completely as we requested, or we fail. We don't return partial data because the caller will almost never find that to be a useful behavior. Specify a makefile knob which users can use to specify one or more suitable CSPRNGs, and turn the multiple string options into a set of defines, since we cannot match on strings in the preprocessor. We allow multiple options to make the job of handling this in autoconf easier. The order of options is important here. On systems with arc4random, which is most of the BSDs, we use that, since, except on MirBSD and macOS, it uses ChaCha20, which is extremely fast, and sits entirely in userspace, avoiding a system call. We then prefer getrandom over getentropy, because the former has been available longer on Linux, and then OpenSSL. Finally, if none of those are available, we use /dev/urandom, because most Unix-like operating systems provide that API. We prefer options that don't involve device files when possible because those work in some restricted environments where device files may not be available. Set the configuration variables appropriately for Linux and the BSDs, including macOS, as well as Windows and NonStop. We specifically only consider versions which receive publicly available security support here. For the same reason, we don't specify getrandom(2) on Linux, because CentOS 7 doesn't support it in glibc (although its kernel does) and we don't want to resort to making syscalls. Finally, add a test helper to allow this to be tested by hand and in tests. We don't add any tests, since invoking the CSPRNG is not likely to produce interesting, reproducible results. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-10Merge branch 'ab/makefile-hook-list-dependency-fix'Libravatar Junio C Hamano1-2/+2
Fix dependency rules to generate hook-list.h header file. * ab/makefile-hook-list-dependency-fix: Makefile: correct the dependency graph of hook-list.h
2022-01-10Merge branch 'ab/makefile-pager-env-is-used-only-by-pager.c'Libravatar Junio C Hamano1-5/+6
* ab/makefile-pager-env-is-used-only-by-pager.c: Makefile: move -DPAGER_ENV from BASIC_CFLAGS to EXTRA_CPPFLAGS
2022-01-10Merge branch 'ab/makefile-msgfmt-wo-stats'Libravatar Junio C Hamano1-1/+1
Make the recipe that runs msgfmt less noisy. * ab/makefile-msgfmt-wo-stats: Makefile: don't invoke msgfmt with --statistics
2022-01-07hook: add 'run' subcommandLibravatar Emily Shaffer1-0/+1
In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event. Most of our hooks require more complex functionality than this, but let's start with the bare minimum required to support our simplest hooks. In terms of implementation the usage_with_options() and "goto usage" pattern here mirrors that of builtin/{commit-graph,multi-pack-index}.c. Some of the implementation here, such as a function being named run_hooks_opt() when it's tasked with running one hook, to using the run_processes_parallel_tr2() API to run with jobs=1 is somewhere between a bit odd and and an overkill for the current features of this "hook run" command and the hook.[ch] API. This code will eventually be able to run multiple hooks declared in config in parallel, by starting out with these names and APIs we reduce the later churn of renaming functions, switching from the run_command() to run_processes_parallel_tr2() API etc. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-25Makefile: don't invoke msgfmt with --statisticsLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Remove the --statistics flag that I added in 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18). Our Makefile output is good about reducing verbosity by default, except in this case: $ rm -rf po/build/locale/e*; time make -j $(nproc) all SUBDIR templates MKDIR -p po/build/locale/el/LC_MESSAGES MSGFMT po/build/locale/el/LC_MESSAGES/git.mo MKDIR -p po/build/locale/es/LC_MESSAGES MSGFMT po/build/locale/es/LC_MESSAGES/git.mo 1038 translated messages, 3325 untranslated messages. 5230 translated messages. I didn't have any good reason for using --statistics at the time other than ad-hoc eyeballing of the output. We don't need to spew out exactly how many messages we've got translated every time. Now we'll instead emit: $ rm -rf po/build/locale/e*; time make -j $(nproc) all SUBDIR templates MKDIR -p po/build/locale/el/LC_MESSAGES MSGFMT po/build/locale/el/LC_MESSAGES/git.mo MKDIR -p po/build/locale/es/LC_MESSAGES MSGFMT po/build/locale/es/LC_MESSAGES/git.mo Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-25Makefile: move -DPAGER_ENV from BASIC_CFLAGS to EXTRA_CPPFLAGSLibravatar Ævar Arnfjörð Bjarmason1-5/+6
Remove -DPAGER_ENV from the BASIC_CFLAGS and instead have it passed via the EXTRA_CPPFLAGS passed when compiling pager.c. This doesn't change anything except to make it clear that only pager.c needs this, as it's the only user of this define. See 995bc22d7f8 (pager: move pager-specific setup into the build, 2016-08-04) for the commit that originally added this. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-25Makefile: correct the dependency graph of hook-list.hLibravatar Ævar Arnfjörð Bjarmason1-2/+2
Fix an issue in my cfe853e66be (hook-list.h: add a generated list of hooks, like config-list.h, 2021-09-26), the builtin/help.c was inadvertently made to depend on hook-list.h, but it's used by builtin/bugreport.c. The hook.c also does not depend on hook-list.h. It did in an earlier version of the greater series cfe853e66be was extracted from, but not anymore. We might end up needing that line again, but let's remove it for now. Reported-by: Mike Hommey <mh@glandium.org> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-21Merge branch 'js/scalar'Libravatar Junio C Hamano1-0/+9
Add pieces from "scalar" to contrib/. * js/scalar: scalar: implement the `version` command scalar: implement the `delete` command scalar: teach 'reconfigure' to optionally handle all registered enlistments scalar: allow reconfiguring an existing enlistment scalar: implement the `run` command scalar: teach 'clone' to support the --single-branch option scalar: implement the `clone` subcommand scalar: implement 'scalar list' scalar: let 'unregister' handle a deleted enlistment directory gracefully scalar: 'unregister' stops background maintenance scalar: 'register' sets recommended config and starts maintenance scalar: create test infrastructure scalar: start documenting the command scalar: create a rudimentary executable scalar: add a README with a roadmap
2021-12-21Merge branch 'jc/c99-var-decl-in-for-loop'Libravatar Junio C Hamano1-0/+1
Weather balloon to find compilers that do not grok variable declaration in the for() loop. * jc/c99-var-decl-in-for-loop: revision: use C99 declaration of variable in for() loop
2021-12-15Merge branch 'hn/reftable'Libravatar Junio C Hamano1-3/+50
The "reftable" backend for the refs API, without integrating into the refs subsystem, has been added. * hn/reftable: Add "test-tool dump-reftable" command. reftable: add dump utility reftable: implement stack, a mutable database of reftable files. reftable: implement refname validation reftable: add merged table view reftable: add a heap-based priority queue for reftable records reftable: reftable file level tests reftable: read reftable files reftable: generic interface to tables reftable: write reftable files reftable: a generic binary tree implementation reftable: reading/writing blocks Provide zlib's uncompress2 from compat/zlib-compat.c reftable: (de)serialization for the polymorphic record type. reftable: add blocksource, an abstraction for random access reads reftable: utility functions reftable: add error related functionality reftable: add LICENSE hash.h: provide constants for the hash IDs
2021-12-10Merge branch 'bc/require-c99'Libravatar Junio C Hamano1-1/+1
Weather balloon to break people with compilers that do not support C99. * bc/require-c99: git-compat-util: add a test balloon for C99 support
2021-12-04scalar: create a rudimentary executableLibravatar Johannes Schindelin1-0/+9
The idea of Scalar (https://github.com/microsoft/scalar), and before that, of VFS for Git, has always been to prove that Git _can_ scale, and to upstream whatever strategies have been demonstrated to help. With this patch, we start the journey from that C# project to move what is left to Git's own `contrib/` directory, reimplementing it in pure C, with the intention to facilitate integrating the functionality into core Git all while maintaining backwards-compatibility for existing Scalar users (which will be much easier when both live in the same worktree). It has always been the plan to contribute all of the proven strategies back to core Git. For example, while the virtual filesystem provided by VFS for Git helped the team developing the Windows operating system to move onto Git, while trying to upstream it we realized that it cannot be done: getting the virtual filesystem to work (which we only managed to implement fully on Windows, but not on, say, macOS or Linux), and the required server-side support for the GVFS protocol, made this not quite feasible. The Scalar project learned from that and tackled the problem with different tactics: instead of pretending to Git that the working directory is fully populated, it _specifically_ teaches Git about partial clone (which is based on VFS for Git's cache server), about sparse checkout (which VFS for Git tried to do transparently, in the file system layer), and regularly runs maintenance tasks to keep the repository in a healthy state. With partial clone, sparse checkout and `git maintenance` having been upstreamed, there is little left that `scalar.exe` does which `git.exe` cannot do. One such thing is that `scalar clone <url>` will automatically set up a partial, sparse clone, and configure known-helpful settings from the start. So let's bring this convenience into Git's tree. The idea here is that you can (optionally) build Scalar via make -C contrib/scalar/ This will build the `scalar` executable and put it into the contrib/scalar/ subdirectory. The slightly awkward addition of the `contrib/scalar/*` bits to the top-level `Makefile` are actually really required: we want to link to `libgit.a`, which means that we will need to use the very same `CFLAGS` and `LDFLAGS` as the rest of Git. An early development version of this patch tried to replicate all the conditional code in `contrib/scalar/Makefile` (e.g. `NO_POLL`) just like `contrib/svn-fe/Makefile` used to do before it was retired. It turned out to be quite the whack-a-mole game: the SHA-1-related flags, the flags enabling/disabling `compat/poll/`, `compat/regex/`, `compat/win32mmap.c` & friends depending on the current platform... To put it mildly: it was a major mess. Instead, this patch makes minimal changes to the top-level `Makefile` so that the bits in `contrib/scalar/` can be compiled and linked, and adds a `contrib/scalar/Makefile` that uses the top-level `Makefile` in a most minimal way to do the actual compiling. Note: With this commit, we only establish the infrastructure, no Scalar functionality is implemented yet; We will do that incrementally over the next few commits. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-03revision: use C99 declaration of variable in for() loopLibravatar Junio C Hamano1-0/+1
There are certain C99 features that might be nice to use in our code base, but we've hesitated to do so in order to avoid breaking compatibility with older compilers. But we don't actually know if people are even using pre-C99 compilers these days. One way to figure that out is to introduce a very small use of a feature, and see if anybody complains, and we've done so to probe the portability for a few features like "trailing comma in enum declaration", "designated initializer for struct", and "designated initializer for array". A few years ago, we tried to use a handy for (int i = 0; i < n; i++) use(i); to introduce a new variable valid only in the loop, but found that some compilers we cared about didn't like it back then. Two years is a long-enough time, so let's try it again. If this patch can survive a few releases without complaint, then we can feel more confident that variable declaration in for() loop is supported by the compilers our user base use. And if we do get complaints, then we'll have gained some data and we can easily revert this patch. Helped-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-01git-compat-util: add a test balloon for C99 supportLibravatar brian m. carlson1-1/+1
The C99 standard was released in January 1999, now 22 years ago. It provides a variety of useful features, including variadic arguments for macros, declarations after statements, designated initializers, and a wide variety of other useful features, many of which we already use. We'd like to take advantage of these features, but we want to be cautious. As far as we know, all major compilers now support C99 or a later C standard, such as C11 or C17. POSIX has required C99 support as a requirement for the 2001 revision, so we can safely assume any POSIX system which we are interested in supporting has C99. Even MSVC, long a holdout against modern C, now supports both C11 and C17 with an appropriate update. Moreover, even if people are using an older version of MSVC on these systems, they will generally need some implementation of the standard Unix utilities for the testsuite, and GNU coreutils, the most common option, has required C99 since 2009. Therefore, we can safely assume that a suitable version of GCC or clang is available to users even if their version of MSVC is not sufficiently capable. Let's add a test balloon to git-compat-util.h to see if anyone is using an older compiler. We'll add a comment telling people how to enable this functionality on GCC and Clang, even though modern versions of both will automatically do the right thing, and ask people still experiencing a problem to report that to us on the list. Note that C89 compilers don't provide the __STDC_VERSION__ macro, so we use a well-known hack of using "- 0". On compilers with this macro, it doesn't change the value, and on C89 compilers, the macro will be replaced with nothing, and our value will be 0. For sparse, we explicitly request the gnu99 style because we've traditionally taken advantage of some GCC- and clang-specific extensions when available and we'd like to retain the ability to do that. sparse also defaults to C89 without it, so things will fail for us if we don't. Update the cmake configuration to require C11 for MSVC. We do this because this will make MSVC to use C11, since it does not explicitly support C99. We do this with a compiler options because setting the C_STANDARD option does not work in our CI on MSVC and at the moment, we don't want to require C11 for Unix compilers. In the Makefile, don't set any compiler flags for the compiler itself, since on some systems, such as FreeBSD, we actually need C11, and asking for C99 causes things to fail to compile. The error message should make it obvious what's going wrong and allow a user to set the appropriate option when building in the event they're using a Unix compiler that doesn't support it by default. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-29Merge branch 'ab/sh-retire-helper-functions'Libravatar Junio C Hamano1-16/+10
Make a few helper functions unused and then lose them. * ab/sh-retire-helper-functions: git-sh-setup: remove "sane_grep", it's not needed anymore git-sh-setup: remove unused sane_egrep() function git-instaweb: unconditionally assume that gitweb is mod_perl capable Makefile: remove $(NO_CURL) from $(SCRIPT_DEFINES) Makefile: remove $(GIT_VERSION) from $(SCRIPT_DEFINES) Makefile: move git-SCRIPT-DEFINES adjacent to $(SCRIPT_DEFINES)
2021-10-29Merge branch 'ab/make-sparse-for-real'Libravatar Junio C Hamano1-1/+1
Fix-up for a recent topic. * ab/make-sparse-for-real: Makefile: remove redundant GIT-CFLAGS dependency from "sparse"
2021-10-21Makefile: remove redundant GIT-CFLAGS dependency from "sparse"Libravatar Ævar Arnfjörð Bjarmason1-1/+1
The "sparse" target needed the GIT-CFLAGS dependency before my c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23), but since then it depends on the corresponding *.o files, which in turn depend on the correct header files, as well as on GIT-CFLAGS. There's no need to re-state this dependency here. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21git-sh-setup: remove "sane_grep", it's not needed anymoreLibravatar Ævar Arnfjörð Bjarmason1-5/+1
Remove the sane_grep() shell function in git-sh-setup. The two reasons for why it existed don't apply anymore: 1. It was added due to GNU grep supporting GREP_OPTIONS. See e1622bfcbad (Protect scripted Porcelains from GREP_OPTIONS insanity, 2009-11-23). Newer versions of GNU grep ignore that, but even on older versions its existence won't matter, none of these sane_grep() uses care about grep's output, they're merely using it to check if a string exists in a file or stream. We also don't care about the "LC_ALL=C" that "sane_grep" was using, these greps for fixed or ASCII strings will behave the same under any locale. 2. The SANE_TEXT_GREP added in 71b401032b9 (sane_grep: pass "-a" if grep accepts it, 2016-03-08) isn't needed either, none of these grep uses deal with binary data. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21Makefile: remove $(NO_CURL) from $(SCRIPT_DEFINES)Libravatar Ævar Arnfjörð Bjarmason1-2/+1
Stop including $(NO_CURL) in $(SCRIPT_DEFINES). The "@NO_CURL@" replacement added in 6c5c62f3401 (Print an error if cloning a http repo and NO_CURL is set, 2006-02-15) has not been referenced by anything in-tree since 49eb8d39c78 (Remove contrib/examples/*, 2018-03-25). That commit removed the reference from contrib/examples/*, but this @@NO_CURL@@ hasn't been used since git-pull.sh was the primary entry point for "git pull". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21Makefile: remove $(GIT_VERSION) from $(SCRIPT_DEFINES)Libravatar Ævar Arnfjörð Bjarmason1-1/+1
Remove the $(GIT_VERSION) from $(SCRIPT_DEFINES). Now every time HEAD changes in a development copy we don't need to re-build the scripts and script libraries. This has not been needed since 2b9391bc675 (Makefile: do not replace @@GIT_VERSION@@ in shell scripts, 2012-06-20). On my setup this changes the re-making of 44 targets in a development copy where moved HEAD to 27. The $(GIT_VERSION) was seemingly left here by mistake or omission. We didn't need it since 2b9391bc675, but in the later e4dd89ab984 (Makefile: update scripts when build-time parameters change, 2012-06-20) it was added to SCRIPT_DEFINES. The two were part of the same series of patches, and given the summary in [1] and [2] it looks like this was probably a case of some earlier version of a later patch being combined with an updated earlier patch. 1. https://lore.kernel.org/git/20120619232231.GA6328@sigill.intra.peff.net/ 2. https://lore.kernel.org/git/20120619232453.GB6496@sigill.intra.peff.net/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21Makefile: move git-SCRIPT-DEFINES adjacent to $(SCRIPT_DEFINES)Libravatar Ævar Arnfjörð Bjarmason1-8/+7
When "GIT-SCRIPT-DEFINES" was added in e4dd89ab984 (Makefile: update scripts when build-time parameters change, 2012-06-20) the rules for generating the scripts themselves were moved further away from the "cmd_munge_script" added in 46bac904581 (Do not install shell libraries executable, 2010-01-31). Let's move these around so that the variables and defines needed by given targets immediately precede them. This is not needed for any subsequent changes to work, but makes the code consistent with how GIT-PERL-DEFINES is structured. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-18Merge branch 'js/retire-preserve-merges'Libravatar Junio C Hamano1-2/+0
The "--preserve-merges" option of "git rebase" has been removed. * js/retire-preserve-merges: sequencer: restrict scope of a formerly public function rebase: remove a no-longer-used function rebase: stop mentioning the -p option in comments rebase: remove obsolete code comment rebase: drop the internal `rebase--interactive` command git-svn: drop support for `--preserve-merges` rebase: drop support for `--preserve-merges` pull: remove support for `--rebase=preserve` tests: stop testing `git rebase --preserve-merges` remote: warn about unhandled branch.<name>.rebase values t5520: do not use `pull.rebase=preserve`
2021-10-13Merge branch 'ab/make-sparse-for-real'Libravatar Junio C Hamano1-3/+6
Prevent "make sparse" from running for the source files that haven't been modified. * ab/make-sparse-for-real: Makefile: make the "sparse" target non-.PHONY
2021-10-13Merge branch 'ab/config-based-hooks-1'Libravatar Junio C Hamano1-13/+15
Mostly preliminary clean-up in the hook API. * ab/config-based-hooks-1: hook-list.h: add a generated list of hooks, like config-list.h hook.c users: use "hook_exists()" instead of "find_hook()" hook.c: add a hook_exists() wrapper and use it in bugreport.c hook.[ch]: move find_hook() from run-command.c to hook.c Makefile: remove an out-of-date comment Makefile: don't perform "mv $@+ $@" dance for $(GENERATED_H) Makefile: stop hardcoding {command,config}-list.h Makefile: mark "check" target as .PHONY
2021-10-12Merge branch 'ab/make-clean-depend-dirs' into maintLibravatar Junio C Hamano1-1/+0
"make clean" has been updated to remove leftover .depend/ directories, even when it is not told to use them to compute header dependencies. * ab/make-clean-depend-dirs: Makefile: clean .depend dirs under COMPUTE_HEADER_DEPENDENCIES != yes
2021-10-12Merge branch 'ab/no-more-check-bindir' into maintLibravatar Junio C Hamano1-2/+1
Build simplification. * ab/no-more-check-bindir: Makefile: remove the check_bindir script
2021-10-12Merge branch 'sg/make-fix-ar-invocation' into maintLibravatar Junio C Hamano1-2/+2
Build fix. * sg/make-fix-ar-invocation: Makefile: remove archives before manipulating them with 'ar'
2021-10-11Merge branch 'ab/sanitize-leak-ci'Libravatar Junio C Hamano1-0/+5
CI learns to run the leak sanitizer builds. * ab/sanitize-leak-ci: tests: add a test mode for SANITIZE=leak, run it in CI Makefile: add SANITIZE=leak flag to GIT-BUILD-OPTIONS
2021-10-08Add "test-tool dump-reftable" command.Libravatar Han-Wen Nienhuys1-0/+1
This command dumps individual tables or a stack of of tables. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: implement stack, a mutable database of reftable files.Libravatar Han-Wen Nienhuys1-0/+1
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: implement refname validationLibravatar Han-Wen Nienhuys1-0/+1
The packed/loose format has restrictions on refnames: a and a/b cannot coexist. This limitation does not apply to reftable per se, but must be maintained for interoperability. This code adds validation routines to abort transactions that are trying to add invalid names. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: add merged table viewLibravatar Han-Wen Nienhuys1-0/+2
This adds an abstract, read-only interface to the ref database. This primitive is used to construct the read view of the ref database (the read view is constructed by merging several *.ref files). It also provides the mechanism to provide a unified view of the refs in the main repository and the per-worktree refs. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: add a heap-based priority queue for reftable recordsLibravatar Han-Wen Nienhuys1-0/+2
This is needed to create a merged view multiple reftables Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: reftable file level testsLibravatar Han-Wen Nienhuys1-0/+1
With support for reading and writing files in place, we can construct files (in memory) and attempt to read them back. Because some sections of the format are optional (eg. indices, log entries), we have to exercise this code using multiple sizes of input data Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: read reftable filesLibravatar Han-Wen Nienhuys1-0/+2
This supports reading a single reftable file. The commit introduces an abstract iterator type, which captures the usecases both of reading individual refs, and iterating over a segment of the ref namespace. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: generic interface to tablesLibravatar Han-Wen Nienhuys1-0/+3
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08reftable: write reftable filesLibravatar Han-Wen Nienhuys1-0/+1
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>