#Tag · bonfire.cafe

While working on our project, we occasionally have to do a full checkout of the whole codebase, which is several gigabytes. Over time, this operation has gotten very, very, very slow - I mean "2+ hours to perform a checkout" slow.

This was getting quite ridiculous. Even though it's CVS, it shouldn't crawl like this. A quick build of CVS with debug symbols and sampling the "cvs server" process with Linux perf showed something peculiar: The code was spending the majority of the time inside one function.

So what is this get_memnode() function? Turns out this is a support function from Gnulib that enables page-aligned memory allocations. (NOTE: I have no clue why CVS thinks doing page-aligned allocations is beneficial here - but here we are.)

The code in question has support for three different backend allocators:
1. mmap
2. posix_memalign
3. malloc

Sounds nice, except that both 1 and 3 use a linked list to track the allocations. The get_memnode() function is called when deallocating memory to find out the original pointer to pass to the backend deallocation function: The node search code appears as:

for (c = *p_next; c != NULL; p_next = &c->next, c = c->next)
if (c->aligned_ptr == aligned_ptr)
break;

The get_memnode() function is called from pagealign_free():

#if HAVE_MMAP
if (munmap (aligned_ptr, get_memnode (aligned_ptr)) < 0)
error (EXIT_FAILURE, errno, "Failed to unmap memory");
#elif HAVE_POSIX_MEMALIGN
free (aligned_ptr);
#else
free (get_memnode (aligned_ptr));
#endif

This is an O(n) operation. CVS must be allocating a huge number of small allocations, which will result in it spending most of the CPU time in get_memnode() trying to find the node to remove from the list.

Why should we care? This is "just CVS" after all. Well, Gnulib is used in a lot of projects, not just CVS. While pagealign_alloc() is likely not the most used functionality, it can still end up hurting performance in many places.

The obvious easy fix is to prefer the posix_memalign method over the other options (I quickly made this happen for my personal CVS build by adding tactical #undef HAVE_MMAP). Even better, the list code should be replaced with something more sensible. In fact, there is no need to store the original pointer in a list; a better solution is to allocate enough memory and store the pointer before the calculated aligned pointer. This way, the original pointer can be fetched from the negative offset of the pointer passed to pagealign_free(). This way, it will be O(1).

I tried to report this to the Gnulib project, but I have trouble reaching gnu.org services currently. I'll be sure to do that once things recover.

#opensource #development #bugstories

pagealign_free() function calling get_memnode() for other than HAVE_POSIX_MEMALIGN code paths.

get_memnode function code using linear list search, which is O(n).

perf report revealing ton of CPU time spent in get_memnode() traversing a linked list.

Harry Sintonen

@harrysintonen@infosec.exchange · 2 months ago

As it happens, we still use CVS in our operating system project (there are reasons for doing this, but migration to git would indeed make sense).

The code in question has support for three different backend allocators:
1. mmap
2. posix_memalign
3. malloc

for (c = *p_next; c != NULL; p_next = &c->next, c = c->next)
if (c->aligned_ptr == aligned_ptr)
break;

The get_memnode() function is called from pagealign_free():

I tried to report this to the Gnulib project, but I have trouble reaching gnu.org services currently. I'll be sure to do that once things recover.

#opensource #development #bugstories

alcinnz boosted

alcinnz

@alcinnz@floss.social · 2 months ago

I'd track a stack of control-flow preprocessor lines tracking whether to keep or discard the lines between them, inserting #line where needed. This is how I'd handle #if, #elif, #else, #endif, #ifdef, #ifndef, #elifdef, #elifndef.

Some of these take identifiers whose presence it should check in the macros table, others would interpret infix expressions via a couple stacks & The Shunting Yard Algorithm. Or they simply end a control-flow block.

#undef removes an entry from the macros table.

3/4

alcinnz

@alcinnz@floss.social · 2 months ago

#include would read the named file & concatenate into the output string preceded by a #line directive. #embed would work similarly, but reformats the read bytes into hexadecimal literals.

#define would extract the identifier & parse the following optional argument list & body removing (escaped) newlines to load into a "macro" table.

Non-preprocessor lines would be scanned for these macros' identifiers to perform a find & replace, recursing to handle substitute in parameters.

2/3?

alcinnz

@alcinnz@floss.social replied · 2 months ago

#undef removes an entry from the macros table.

3/4

Michael Dexter

@dexter@bsd.network · 2 months ago

Do you have a favorite method of removing C-style inline and multiline comments from code?

Removing trailing tabs that were before them and resulting double lines is a plus.

Tim Chase

@gumnos@mastodon.bsd.cafe replied · 2 months ago

@dexter

IIUC,

$ gcc -fpreprocessed -dD -E -P file.c > no_comments.c

should do the trick. You really need a proper C/C++ aware lexer/parser because while simple substitutions

:%s@\_s*//.*\n/\r

in Vim can get thrown off by "//" or "/*…*/" embedded in string-constants. There are additional complexities with multi-line strings and nested comments. I'm not sure if you also intend to take into consideration ": #if 0" blocks.

alcinnz boosted

Peter Hutterer

@whot@floss.social · 3 months ago

libinput 1.29-rc3 is out but this one is a brown paper bag release. A mixup of #if and #ifdef caused events to be debug-logged even if the meson option for it was disabled.

This means passwords may have leaked into your system logs if debug-logging was enabled. See the below link for more details but the TLDR is mutter/wlroots not affected, kwin/Xorg maybe but unlikely.

Either way, if you're on rc1 or rc2, upgrade now. 1.28.x not affected.

https://gitlab.freedesktop.org/libinput/libinput/-/releases/1.28.903

d@nny disc@ mc²

@hipsterelectron@circumstances.run · 3 months ago

and i admired how this comment about not knowing what to do remained even if part of the problem has now been solved

-      /* Default PATH is implementation-defined, so we don't know how
-         to conduct the search.  */
+      /* We don't know how to conduct the search.  */
       return NULL;

d@nny disc@ mc²

@hipsterelectron@circumstances.run replied · 3 months ago

their cgit is going incredibly slowly i presume from corporate inference so here's the diff with the comments lol. love distributed version control systems

commit eb9ec79c13f17d610fcb6de49628b8a7686fbab7
Author: Paul Eggert 
Date: Thu Jul 24 14:49:52 2025 -0700

 PATH defaults now act more like GNU and POSIX

 When PATH is unset or empty, use the system default,
 to be consistent with GNU/Linux and with POSIX.
 If there is no system default do not default to "."
 as that can be dangerous.
 * src/callproc.c (init_callproc_1, init_callproc):
 Default PATH to the null pointer, not the empty string.
 * src/emacs.c (default_PATH): New function.
 (find_emacs_executable, decode_env_path): Be consistent with POSIX
 and with glibc about what to do when PATH is unset or empty.

diff --git a/doc/emacs/cmdargs.texi b/doc/emacs/cmdargs.texi
index 00167056be1..79ae2d064d1 100644
--- a/doc/emacs/cmdargs.texi
+++ b/doc/emacs/cmdargs.texi
@@ -695,6 +695,7 @@ General Variables
 A colon-separated list of directories containing executable files.
 This is used to initialize the variable @code{exec-path}
 (@pxref{Shell}).
+If unset or empty, an implementation-dependent default is used.
 @item PWD
 @vindex PWD@r{, environment variable}
 If set, this should be the default directory when Emacs was started.
diff --git a/doc/emacs/misc.texi b/doc/emacs/misc.texi
index 88b7bfc7049..2c2c710eea4 100644
--- a/doc/emacs/misc.texi
+++ b/doc/emacs/misc.texi
@@ -753,8 +753,8 @@ Shell
 (either in the @var{cmd} argument to one of the above commands, or in
 other contexts), Emacs searches for the program in the directories
 specified by the variable @code{exec-path}. The value of this
-variable must be a list of directories; the default value is
-initialized from the environment variable @env{PATH} when Emacs is
+variable must be a list of directories; the default value
+depends on the environment variable @env{PATH} when Emacs is
 started (@pxref{General Variables}).

 @kbd{M-x eshell} invokes a shell implemented entirely in Emacs. It
diff --git a/etc/NEWS b/etc/NEWS
index 865e5e6b6d5..8f396403ba2 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -2755,6 +2755,12 @@ option, but can optionally return the equivalent of 'exec-suffixes' from
 a remote host. It must be used in conjunction with the function
 'exec-path'.

++++
+** The 'exec-path' variable now uses same default PATH as other programs.
+That is, if the PATH environment variable is unset or empty, 'exec-path'
+now acts as if PATH is the system default, which is "/bin:/usr/bin"
+on GNU/Linux systems.
+

 * Changes in Emacs 31.1 on Non-Free Operating Systems

diff --git a/src/callproc.c b/src/callproc.c
index 7059a2bac6f..e1a369b47f4 100644
--- a/src/callproc.c
+++ b/src/callproc.c
@@ -1960,7 +1960,7 @@ init_callproc_1 (void)
 Vexec_path = decode_env_path ("EMACSPATH", PATH_EXEC, 0);
 Vexec_directory = Ffile_name_as_directory (Fcar (Vexec_path));
 /* FIXME? For ns, path_exec should go at the front? /
- Vexec_path = nconc2 (decode_env_path ("PATH", "", 0), Vexec_path);
+ Vexec_path = nconc2 (decode_env_path ("PATH", NULL, 0), Vexec_path);
 }

 / This is run after init_cmdargs, when Vinstallation_directory is valid. /
@@ -1985,7 +1985,7 @@ init_callproc (void)
 {
 / Running uninstalled, so default to tem rather than PATH_EXEC. */
 Vexec_path = decode_env_path ("EMACSPATH", SSDATA (tem), 0);
- Vexec_path = nconc2 (decode_env_path ("PATH", "", 0), Vexec_path);
+ Vexec_path = nconc2 (decode_env_path ("PATH", NULL, 0), Vexec_path);
 }

 Vexec_directory = Ffile_name_as_directory (tem);
diff --git a/src/emacs.c b/src/emacs.c
index 3689c92c8b2..aa762e7edb1 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -737,6 +737,44 @@ argmatch (char **argv, int argc, const char *sstr, const char lstr,
 }
 }

+/ Return the default PATH if it can be determined, NULL otherwise. */
+
+static char const *
+default_PATH (void)
+{
+ static char const path;
+
+ / A static buffer big enough so that confstr is called just once
+ in GNU/Linux, where the default PATH is "/bin:/usr/bin".
+ If staticbuf[0], path is already initialized. */
+ static char staticbuf[16];
+
+ if (!staticbuf[0])
+ {
+#ifdef _CS_PATH
+ char buf = staticbuf;
+ size_t bufsize = sizeof staticbuf, s;
+
+ / If necessary call confstr a second time with a bigger buffer. /
+ while (bufsize < (s = confstr (_CS_PATH, buf, bufsize)))
+ {
+ buf = xmalloc (s);
+ bufsize = s;
+ }
+
+ if (s == 0)
+ {
+ staticbuf[0] = 1;
+ buf = NULL;
+ }
+
+ path = buf;
+#endif
+ }
+
+ return path;
+}
+
 #if !defined HAVE_ANDROID || defined ANDROID_STUBIFY

 / Find a name (absolute or relative) of the Emacs executable whose
@@ -778,10 +816,11 @@ find_emacs_executable (char const *argv0, ptrdiff_t *candidate_size)
 ptrdiff_t argv0_length = strlen (argv0);

 const char *path = getenv ("PATH");
+ if (! (path && path))
+ path = default_PATH ();
 if (!path)
 {
- / Default PATH is implementation-defined, so we don't know how
- to conduct the search. /
+ / We don't know how to conduct the search. */
 return NULL;
 }

@@ -3217,15 +3256,19 @@ decode_env_path (const char *evarname, const char *defalt, bool empty)
 to initialize variables when Emacs starts up, and isn't called
 after that. */
 if (evarname != 0)
- path = getenv (evarname);
+ {
+ path = getenv (evarname);
+ if (! (path && *path) && strcmp (evarname, "PATH") == 0)
+ path = default_PATH ();
+ }
 else
 path = 0;
 if (!path)
 {
-#ifdef NS_SELF_CONTAINED
- path = ns_relocate (defalt);
-#else
 path = defalt;
+#ifdef NS_SELF_CONTAINED
+ if (path)
+ path = ns_relocate (path);
 #endif
 #ifdef WINDOWSNT
 defaulted = 1;
@@ -3277,7 +3320,7 @@ decode_env_path (const char *evarname, const char *defalt, bool empty)
 }
 #endif
 lpath = Qnil;
- while (1)
+ for (; path; path = *p ? p + 1 : NULL)
 {
 p = strchr (path, SEPCHAR);
 if (!p)
@@ -3320,10 +3363,6 @@ decode_env_path (const char *evarname, const char defalt, bool empty)
 } / !NILP (element) */

 lpath = Fcons (element, lpath);
- if (*p)
- path = p + 1;
- else
- break;
 }

 return Fnreverse (lpath);

Peter Hutterer

@whot@floss.social · 3 months ago

libinput 1.29-rc3 is out but this one is a brown paper bag release. A mixup of #if and #ifdef caused events to be debug-logged even if the meson option for it was disabled.

Either way, if you're on rc1 or rc2, upgrade now. 1.28.x not affected.

https://gitlab.freedesktop.org/libinput/libinput/-/releases/1.28.903

Thomas Ricouard

@dimillian@mastodon.social · 4 months ago

@ombremad It's mostly additive, so apart from a few #if iOS 26 wrappers, nothing crazy