Oily Pine 2 - Throwing log-files at ChatGPT

Oily Pine 2 - Throwing log-files at ChatGPT
Author: Samuel Hierholzer	Date: 01 Jan 2026

This is the second article in a miniseries of 3 Blog posts:

Building Alpine Packages to test a shell
Throwing log-files at ChatGPT?
Building packages reliably (To be released)

Intro

I started the Oily Pine project in a try to build all Alpine Linux Packages using OSH as shell.

In the last article I’ve shown how to set up a build environment and finished the article with a first run. This first run resulted in 5327 successfully built packages until the disk filled up, with 1173 failed packages. Since we want to make OSH behave like Busybox Ash, we only care about the failed packages.

Analyzing 1173 failed package builds manually one by one is a bit much, though. As visible in this walk-through of a bugfix by Aidan Olsen, the procedure can take a few hours for a single package.

Luckily, we can assume that many bugs occur in more than just one package. Briefly looking into just a few logs I could confirm that some errors seemed to happen repeatedly. So instead of jumping straight into deep analysis package by package, we can skim the logs and categorize the errors. In the best case we’ll only need to look into a single log-file per category to know how all of them failed. And we can start with the most prevalent bugs first to get the number down as fast as possible. But even if we only need half a minute to categorize a log-file, it’s still almost 10 hours of staring at log files.

But what could be better to churn through so much text and categorize it? A machine?… AI?! I’ve only used LLMs for output that was verifiable code or translations, but this time the LLM would be “the program” (instead of writing the program)!

Let’s try.

Talking to ChatGPT

So I wrote a small script to send the last 40 lines of each log-file to ChatGPT and try to summarize the problem in maximum 5 words. I wrote the script in YSH, because it has a few features which made the below 16 lines of code a bit extra nice. Since the whole project is about the Oils project, I also like to give you a bit of a showcase/intro into the non-POSIX compliant part of Oils. I really believe in it as a path forward from the minefield that POSIX shell is.

# simplified categorization-script

#!/usr/bin/env ysh

# *generate a list of log files with errors from the builrepo log files*
var bad_log_files = glob("/path/to/folder/with/failed/logs/*")

for file in (bad_log_files) {
  ... cat ./PROMPT_TEMPLATE.md
      # Ignore all lines from abuild removing packages at the end
    <(... grep -v "Purging" $file | head -n -1
      # the error should be in the last few lines.
      # chatgpt can't handle too much input!
      | tail -40
      ;)
    | chatgpt
    | tr -d u'\n'
    >> $out_file;

  # Add a newline
  echo >> $out_file
}

The first line shows us how variable assignments work in YSH:

var bad_log_files = glob("/path/to/folder/with/failed/logs/*")

In bash, it would be bad_log_files=("/path/to/folder/with/failed/logs/"*) (note the * being outside the quotes). In POSIX the only sane way is to not store it in a variable: for file in /path/to/folder/with/failed/logs/*. At a first glance YSH looks just more verbose. But what actually happens is that after the = YSH enters an expression-mode, which has functions (like glob) and is dynamically typed like Python or JavaScript. Unquoted globs are not possible in that expression mode, because it would conflict with mathematical multiplication (e.g. var x = 5*5 is 25, NOT a list of files starting and ending with a 5).

The expression mode is probably the biggest difference from POSIX shell to YSH. At first it seems somewhat random and hard to understand when YSH switches from shell-like “Command Mode” to the “Expression Mode”, but it doesn’t require much practice to become an intuition. The Vim Syntax Highlighter can also be a big help, as it changes the color for the expression mode. (A Pygments highlighter I could use for this blog is still an open to-do).

Now let’s look at the for loop:

for file in (bad_log_files) {

}

In YSH that’s how a typical for loop looks like. Inside of the parentheses () YSH enters expression mode again and expects either a list or dict from the expression.

Another option would be to stay in the command mode and use for file in @bad_log_files { ... }. This @mylist syntax comes from Perl - though YSH only knows the two sigils $ and @ (and they use the same namespace). @bad_log_files is roughly comparable to "${bad_log_files[@]}" and adds all list elements as arguments (without word splitting!).

This also shows the backwards-compatibility of YSH and the focus on only changing/extending where necessary. It’s still perfectly valid to use a POSIX test like if test -f myfile.

The two ... in the following block allow us to define a single command spanning across multiple lines. This spares us the ` \ ` at the end of each line and allows us to put comments in between:

  ... cat ./PROMPT_TEMPLATE.md
      # Ignore all lines from abuild removing packages at the end
    <(... grep -v "Purging" $file | head -n -1
      # the error should be in the last few lines.
      # chatgpt can't handle too much input!
      | tail -40
      ;)
    | chatgpt
    | tr -d u'\n'
    >> $out_file;

So these lines are really just one command-chain, including a subcommand. It does the exact same thing as this POSIX-compatible one-liner:

cat ./PROMPT_TEMPLATE.md <(grep -v "Purging" $file | head -n -1 | tail -40) | chatgpt | tr -d u'\n' >> $out_file;

The command chain does the following:

Concatenate the PROMPT_TEMPLATE.md and the output of the following subshell:
- Remove all lines saying Purging from the log-file (The logs usually ended with many Purging BUILD-DEPENDENCY lines)
- Remove everything except for the last 40 lines
Feed it to ChatGPT
Remove any newlines in the ChatGPT output (which shouldn’t be more than 5 words anyway!!1! 😉)

On this note; I used a chatgpt-cli written in Golang, which I installed with the following command:

curl -L -o chatgpt https://github.com/kardolus/chatgpt-cli/releases/latest/download/chatgpt-linux-amd64 && chmod +x chatgpt && sudo mv chatgpt /usr/local/bin/

That’s really not how I usually install software (especially when it comes to LLM stuff), and I urge you to not just install random binaries from GitHub. But after trying and failing with a few other ChatGPT (and Claude) CLI tools, this worked. And so I just stuck with it for that experiment. I did that in the spring of 2025, on an EOL Debian, so there are definitely better ways to access an LLM from the command line. <br > ChatGPT also wants an access token, so I logged into the OpenAI API Platform and created a token there. I like that it doesn’t have a subscription fee but a pay-per use model (and the ability to set a monthly limit).

Regarding the PROMPT_TEMPLATE.md; I’m not (yet) a prompt Engineer. So I copied a prompt from a random blog post and quickly adjusted it to fit my needs. It’s not good, and I didn’t even test if a single sentence instruction would’ve been enough. I just pretended that using an LLM correctly requires a long prompt:

# IDENTITY and CATEGORY
You are a genius error message categorization expert and you are able to understand and categorize errors from software being compiled for Linux.
You specialize in extracting the relevant error messages from a log file and create a very brief category for the error.
Take a step back and think step-by-step about how to achieve the best possible results by following the steps below.

You have no knowledge on what to do with the error message. Your only purpose is to categorize it.

# STEPS
Read the entire log file from an attempted package compilation, trying to identify the exact log line containing the error message which causes the build to fail.
Come up with a description of this error type which is not more than 5 words, to be used as CATEGORY-DESCRIPTION.

# OUTPUT INSTRUCTIONS
Only output a single line of plain text.

Do not give warnings or notes; only output the requested 5 words.
Do not try to give a solution to the error.
You can only output a single sentence of 5 words.
Ensure you follow ALL these instructions when creating your output.

# INPUT
Logs:

And it worked! (mostly)

The output

I only categorized 506 logs from the main repo to not spend too much money initially. Then I created some stats to see how often ChatGPT used the same “5” words. I won’t spare the details, but you’re free to just scroll over it:

c compiler cannot create executables.
phdr segment not covered by load segment.
phdr segment not covered error.
compiler-executable creation error.
c compiler unable to create executables.
c compiler cannot create executables
compiler cannot compile programs.
builddeps failed due to conflicts.
phdr segment not covered by load segment
linker input file not found.
dependency conflict - build failed.
compiler gcc cannot compile programs.
compiler-executable creation error
c compiler executables creation failure
ld returned 1 exit status.
compiler cannot compile programs error.
c compiler executables creation failed.
build dependencies selection failed.
phdr segment error during linking.
linking error during compilation detected
dependency selection error during build.
dependency selection error during build
c++ compiler cannot create executables.
builddeps selection error due to dependencies.
build dependencies selection failure.
build dependencies failed due to conflict.
phdr segment not covered by load.
phdr segment coverage error during build
package dependencies validation failed.
missing dependency resolving error.
linker segment load error unresolved.
linker segmentation permissions error.
linker error - build failed.
gcc compilation failure detected
dependency selection failed for builddeps.
dependency resolution error: package conflict.
compiler unable to create executables.
compiler failed to generate executable.
c compiler fails to compile.
c compiler execution failure error
c compiler execution failure detected
c compiler execution error - build failed.
builddeps selection error: dependencies conflict.
builddeps resolution failure due to conflicts.
unpack failure - checksum mismatch error.
undefined reference in linking process.
undeclared functions causing compilation failure.
unable to detect lua library.
this program requires newer bash.
this program requires bash 4.0.
the compiler failed to generate an executable file.
test suite failure, check error.
test suite check failed.
tests exit prematurely, check failed.
test failure during check.
test case premature exit failure
test case assertion error.
test and dependency handling errors.
static 'setns' redefined, build failed.
simple c program not found.
rwx permissions on load segment.
required symbol 'yaml_document_initialize' not found.
redefinition and conflicting types build error.
prepare error - autopoint failed.
premature exit; test case error.
phdr segment uncovered by load.
phdr segment missing in link.
phdr segment load error - build failure.
phdr segment error in ld
phdr segment error during build.
phdr segment configuration build failure
perl-tk build failed.
packages selection conflict detected: builddeps_failed
package dependencies validation error.
openssl linking error during build.
no error causing the build fail.
no 16 bit type error
missing stdint header in code.
missing makefile, build process failed.
mariadb: fetch failed from server.
makefile not found build error.
makefile error - build failed.
makefile check failure error
make: check failed with errors.
load segment permissions error
load segment permissions cause build failure.
load segment permission issue in linking.
linking errors due to undefined references.
linking error in llvm compilation.
linking error during compilation process.
linker segment load error occurred.
linker reference undefined - espk-bld-fd
linker phdr segment error.
linker issue, ld returned error.
linker error - uwsgi build failed.
linker error: undefined reference to functions
linker error - rule not found.
libev requirement not met.
it seems like there were errors during the build process of the mpfr4 library. the errors indicate that the linker input file specified as "\\\-dlt_objdir=".libs/" was not found, resulting in the build failure.to address this issue, you may need to review the build configuration and ensure that the paths and flags provided to the compiler and linker are correct. specifically, check the way the output directory is specified and make sure it aligns with the structure expected by the build system.you may also want to review the makefile or the build scripts to see how the paths and flags are being set and adjust as needed. additionally, checking for any missing or incorrect dependencies that could be causing the linker input file not to be generated properly may be necessary.once you have reviewed and updated the build configuration accordingly, you can retry the build process to see if the issue has been resolved. if you encounter any further difficulties or need additional assistance, feel free to provide more details so i can offer more specific guidance.
it seems like there were errors during the build process for `cgit`. the error message indicates problems with the ld linker related to the segments' permissions. here are a few suggestions to troubleshoot and potentially resolve the issue:1. **check load and phdr segments permissions**: review the linker script, object files, or options used during compilation and linking to ensure that the load segment permissions align with the phdr segment. make sure there are no conflicting permissions defined.2. **investigate ld warnings**: examine any warnings or notes produced by the ld linker. it may provide additional insights into the segmentation issues encountered during linking.3. **review linker options**: verify that the linker options and flags are correctly set during the linking phase. check if there are any conflicting or incorrect options that might cause the segmentation errors.4. **inspect build logs**: review the complete build logs to identify any earlier warnings or errors that could have led to the segmentation issues during linking. addressing any previous build problems might help resolve the current linker errors.5. **consult build dependencies**: ensure that all build dependencies, including libraries and headers, are correctly installed and accessible. sometimes, missing or incompatible dependencies can lead to linker issues.6. **compiler and linker version compatibility**: make sure that the compiler and linker versions are compatible. incompatibilities between the compiler and linker versions could lead to segmentation problems during linking.7. **seek help from the project community**: if troubleshooting the issue locally proves challenging, consider reaching out to the cgit project community or relevant forums for assistance. other developers might have encountered and resolved similar issues.by following these steps and investigating the build process further, you may be able to identify and address the root cause of the segmentation issues encountered during the linking phase of building `cgit`.
it seems like there was an issue during the compilation process, resulting in the termination of the make commands. this can be caused by various factors, such as incorrect configurations, missing dependencies, or errors in the source code.to troubleshoot and resolve this issue, you can try the following steps:1. check for any error messages or warnings displayed during the compilation. these messages can provide clues about what went wrong.2. verify that all dependencies required for the compilation are properly installed on your system. make sure that all the necessary libraries and tools are available.3. review the source code files mentioned in the error messages, such as `_dd_to_di.c` and `_dd_to_usi.c`, to see if there are any syntax errors or other issues in the code.4. check the makefile and ensure that the build configurations are correct. make sure that the paths to include directories and libraries are set up properly.5. clean the build environment by running `make clean` and then try to rebuild the project using `make`.6. if the issue persists, you may need to seek help from the project's community or maintainers for further assistance.by following these steps, you should be able to diagnose and address the compilation issues to successfully build the project. if you encounter any specific errors or need further assistance, feel free to provide more details for a more targeted solution.
it seems like there was an error during the build process of the program "ifupdown". specifically, the error message indicates issues with phdr segment not covered by load segment and a warning about rwx permissions.to address these errors, you may want to review the linker options related to the phdr segment and the permissions of the load segment. additionally, it could be beneficial to check the linker options being used and ensure they are compatible with the target system.if you need further assistance or guidance on resolving these build errors, feel free to provide more information or details about the target system and the build environment.
it looks like there was an error during the installation process of the libxcb package. the error occurred while trying to install the man pages for the libxcb library. specifically, the error is related to the installation of the man pages for the xvmc (x-video motion compensation) extension in xcb (x c bindings) library.to troubleshoot this issue, you may want to check the following:1. verify that the man page files mentioned in the error message exist and have the correct permissions.2. ensure that the paths specified for the installation of the man pages are correct.3. check the permissions and ownership of the directories where the man pages are being installed.4. review the build logs for any additional error messages that might provide more context on the issue.if you continue to encounter issues with the installation, you may need to consult the documentation or support resources for libxcb to troubleshoot this specific problem further.
it looks like there was an error during the building process of the botan library. the error message indicates that there are issues related to the program headers (phdr) and load segments. additionally, there is a warning about a load segment having read, write, and execute (rwx) permissions, which may contribute to the issue.to diagnose and resolve this error, you may need to check the linker script, build configuration, or the way the library is being built. here are some steps you could follow to troubleshoot and potentially fix the issue:1. **check the linker script**: review the linker script used during the build process. ensure that the phdr and load segments are properly defined and that the memory layout is correct.2. **review the build configuration**: double-check the compiler flags, linker options, and any scripts used during the build process. make sure that the options are appropriate for the target platform and that there are no conflicting settings causing the issue.3. **permissions of load segments**: as the warning indicated, having a load segment with rwx permissions can lead to issues. try to adjust the permissions to be more restrictive (e.g., removing the execute permission) if possible and necessary.4. **check for known issues**: search for any known issues or solutions related to the specific compiler, linker, or platform you are using. it's possible that others have encountered and documented similar issues.5. **build environment**: ensure that the build environment and toolchain are set up correctly for the target platform. inconsistent or incorrect configurations can lead to unexpected errors during the build process.6. **rebuild**: if you suspect that the error is due to a transient issue, try cleaning the build artifacts and rebuilding the library. sometimes, a fresh build can resolve intermittent problems.if the issue persists or if you need further assistance, feel free to provide more details about the build environment or any specific configurations you are using. this additional information can help in providing more targeted troubleshooting steps.
it looks like the build process for the `cargo-auditable` project encountered errors and failed. the build script was unable to compile the dependencies `serde`, `proc-macro2`, `semver`, and `thiserror`.additionally, after the build failure, it seems an action to uninstall dependencies was triggered, specifically the uninstallation of `busybox-1.37.0-r12`.if you need help resolving the build errors or troubleshooting the issues with the project, please provide more details or specific error messages so that i can assist you further.
it looks like the build process for `mdadm` encountered errors related to the linker, specifically issues with the phdr segment not being covered by the load segment, and the presence of a load segment with rwx permissions. these errors caused the build to fail for both `mdadm` and `mdmon`.to troubleshoot and resolve these issues, you may consider the following steps:1. **check build dependencies:**   ensure that all necessary dependencies for building `mdadm` are installed and up to date. sometimes missing or outdated dependencies can cause linker issues.2. **review build flags:**   review the linker flags and options used during the build process to ensure they are correct. make sure that the flags for stack protection, format security, and pie are set appropriately.3. **check compiler and linker versions:**   verify that the versions of the compiler (`gcc`) and linker (`ld`) being used are compatible with the build process. incompatibilities between tools can sometimes lead to linker errors.4. **investigate rwx permissions warning:**   the warning about a `load` segment with rwx permissions could indicate a potential security risk. review the linker options and the generated binary to ensure that the permissions are set correctly.5. **inspect binary output:**   after a failed build, it can be helpful to inspect the generated binary (`mdadm` and `mdmon`) to understand the structure and permissions of the segments. tools like `readelf` or `objdump` can be useful for this purpose.6. **look for relevant compiler and linker output:**   examine the full output from the compiler and linker to see if there are any other warnings or errors that might provide additional context about the failure.7. **search for similar issues:**   if the issue persists, consider searching online forums or issue trackers to see if others have encountered similar problems during the build of `mdadm`. solutions or workarounds from the community could help resolve the problem.by systematically addressing each of these points, you may be able to diagnose the cause of the linker errors and successfully build `mdadm` and `mdmon`. if you encounter any specific errors or need further assistance, feel free to provide additional details for tailored support.
it looks like the build for `botan3` failed. the error message indicates that there was an issue with the `botan` binary, specifically related to phdr segment not being covered by load segment and the permissions associated with the load segment.to troubleshoot and resolve this issue, you may need to review the linker options, segment layout, and permissions set for the binary during the linking stage. double-check the linker flags (`-wl` options) and ensure that the binary is being linked correctly with the necessary segments and permissions.if you encounter this error, you may need to adjust the linker options or review the build process to ensure that the binary is being created correctly.if you need further assistance, please provide more details about the build environment, compiler options, and any custom settings that may be affecting the linking process.
invalid word in parsing command.
invalid package selection error.
invalid command parsing error detected.
invalid command line parsing error
incompatible arguments for ac_output.
http 404 fetch error occurred.
gcc cannot compile programs.
file build failed, uninstalling dependencies.
fetch failed: http/1.1 404 not found.
fetch failed, build dependency removal.
failed to generate an executable.
error occurred during the check.
environment does not support soname.
development dependencies selection error
dependency selection error during builddeps.
dependency selection conflict detected
dependency issue with package selection.
dependency conflict with gettext-dev package.
dependency conflict resolution failure error
dependency conflict resolution failure
dependency conflict prevents build process.
dependency conflict in package selection.
dependency conflict found during build.
dependency conflict detected: build failure
dependency conflict detected - build failure.
dependency conflict detected - bd-cnf
dependency conflict - compilation error.
dependency conflict causing build failure.
connection reset during file download.
conflict in build dependencies detected.
compiler program compilation failure
compiler "gcc" not working.
compiler failure - gcc incompatibility
compiler execution error during build
compiler error in building pciutils.
compiler-error: c compiler cannot create executables
compiler error: cannot compile programs
compiler-cannot create executables
compile-error: c compiler cannot create executables
compilation terminated due to input files error.
compilation error; 'jack' build failed.
command 'not found' error during test.
checking simple c program: not found.
check failed, uninstalling dependencies.
c compiler unable to compile.
c compiler inability error - build failure.
c compiler gcc not found.
c compiler failure - build error.
c compiler fails to create executables.
c compiler fails to build.
c compiler execution issue - compilation failure.
c compiler execution failure - build.
c++ compiler execution error - build failed.
c compiler executable creation error
c-compiler cannot create executables.
category-description: phdr segment not covered by load
cannot create file: permission denied.
build failed - error 2.
build failed due to error.
build failed due to configuration.
builddeps failed: unable to select packages.
builddeps failed: gettext-dev conflict.
builddeps failed - dependencies unmet.
builddeps conflict, uninstalling dependencies.
build dependency conflict - compilation error.
build dependencies validation failure error.
build dependencies failed to resolve.
build dependencies conflict error occurred.
bash version requirement not met.
at-spi2-core: build dependencies failed.
amd64 as(1) feature detection enabled.

Well that was… interesting! Firstly, ChatGPT did NOT always listen to my instruction to use only 5 words. My assumption is that the LLM lost the context in the few cases where it produced huge amounts of text. At least some of these logs contained lines with over 1000 characters. I did also notice a log message that gives advice on debugging possible errors. This could cause a prompt injection and thus vastly different LLM output. But upon verification I couldn’t find the advice in any of the extreme outliers.

Looking at the categories themselves it’s obvious that there is something messed up with the compiler in my build environment.

Conclusion

The script worked and categorized my logs… Definitely not as good as manual categorization would’ve been, though. So, am I satisfied with the result?

Yes and no.

Yes because even though I might’ve realized that something is wrong after manually categorizing only a few logs, I can be weirdly stubborn. I could’ve gone on for way too long before finally deciding that I should fix my environment before looking at more logs. It was much more fun to spend a similar amount of time asking the AI instead.
No because the LLM output was… well… very bad. I’m not sure if I could’ve gotten a much better result with a few improvements like feeding the already existing categories, or tweaking the Prompt.

For the lacking quality - and the fact that on the second try there were only 215 failed packages to categorize - I switched to manual categorization after fixing the build environment. It took me around the same time to categorize 215 logs (with a script) as writing the script to generate the categories. So if I ever have to categorize >1000 logs, I’ll definitely try AI again.

The next blog post will focus on what efforts we took to create a reliable and fast build environment - which gives us good and simple metrics. Because Andy Chu secured an NLNet fund to support us in fixing all these issues, bringing Oils closer to being a real alternative Unix System shell.