Tag: linux

  • Using SED to edit a classic book

    Since I discovered that using TTS speeds up my reading of books, I have been using it almost daily for around a decade, and I am very happy with it.

    On my GNU/Linux computer, I use the “Read Aloud, TTS Voice Reader” browser extension to read books.

    Lately, I have developed a growing interest in classic Arabic books about the purification of the soul written by ابن أبي الدنيا Ibn Abi al-Dunya (823–894 CE).

    However, when I downloaded the books and converted them to .txt files for easier use with the extension, a considerable amount of time was lost to the TTS reading the three asterisks and the hadith, chapter, and page numbers, which appear hundreds of times throughout the book.

    See below as an example:


    6 - حَدَّثَنَا أَبُو خَيْثَمَةَ، وَإِسْحَاقُ بْنُ إِسْمَاعِيلَ، قَالَا: حَدَّثَنَا جَرِيرٌ، عَنِ الْأَعْمَشِ، عَنِ الْحَكَمِ بْنِ عُتَيْبَةَ، وَحَبِيبِ بْنِ أَبِي ثَابِتٍ، عَنْ مَيْمُونِ بْنِ أَبِي شَبِيبٍ، عَنْ مُعَاذِ بْنِ جَبَلٍ رَضِيَ اللَّهُ عَنْهُ قَالَ: قُلْتُ: يَا رَسُولَ اللَّهِ، أَنُؤَاخَذُ بِمَا نَقُولُ؟ قَالَ: «ثَكِلَتْكَ أُمُّكَ يَا ابْنَ جَبَلٍ، وَهَلْ يَكُبُّ النَّاسَ فِي النَّارِ عَلَى مَنَاخِرِهِمْ إِلَّا حَصَائِدُ أَلْسِنَتِهِمْ؟» قَالَ حَبِيبٌ فِي هَذَا الْحَدِيثِ: «وَهَلْ تَقُولُ شَيْئًا إِلَّا لَكَ أَوْ عَلَيْكَ»



    * * *



    الحديث: 6 ¦ الجزء: 1 ¦ الصفحة: 46





    * * *



    7 - حَدَّثَنِي حَمْزَةُ بْنُ الْعَبَّاسِ، أَخْبَرَنَا عَبْدَانُ بْنُ عُثْمَانَ، أَخْبَرَنَا عَبْدُ اللَّهِ، أَنَا مَعْمَرٌ، عَنِ الزُّهْرِيِّ، عَنْ عَبْدِ الرَّحْمَنِ بْنِ مَاعِزٍ، عَنْ سُفْيَانَ بْنِ عَبْدِ اللَّهِ الثَّقَفِيِّ قَالَ: قُلْتُ: يَا رَسُولَ اللَّهِ، حَدِّثْنِي بِأَمْرٍ أَعْتَصِمُ بِهِ. قَالَ: " قُلْ: رَبِّيَ اللَّهُ ثُمَّ اسْتَقِمْ ". قَالَ: قُلْتُ: يَا رَسُولَ اللَّهِ، مَا أَخْوَفُ مَا تَخَافُ عَلَيَّ؟ فَأَخَذَ بِلِسَانِهِ ثُمَّ قَالَ: «هَذَا»

    A SED command

    sed '/^\* \* \*$/d; /^الحديث:/d' الصمت_وآداب_اللسان.txt > editedالصمت_وآداب_اللسان.txt

    Explanation:

    • ^\* \* \*$ matches a line that contains exactly * * * (asterisk, space, asterisk, space, asterisk). The asterisks must be escaped with \ because * is a regex special character.
    • /^الحديث:/d deletes lines that start with الحديث:.
    • All other lines (narration chains, texts, and chapter headings) are kept.

    Output


    6 - حَدَّثَنَا أَبُو خَيْثَمَةَ، وَإِسْحَاقُ بْنُ إِسْمَاعِيلَ، قَالَا: حَدَّثَنَا جَرِيرٌ، عَنِ الْأَعْمَشِ، عَنِ الْحَكَمِ بْنِ عُتَيْبَةَ، وَحَبِيبِ بْنِ أَبِي ثَابِتٍ، عَنْ مَيْمُونِ بْنِ أَبِي شَبِيبٍ، عَنْ مُعَاذِ بْنِ جَبَلٍ رَضِيَ اللَّهُ عَنْهُ قَالَ: قُلْتُ: يَا رَسُولَ اللَّهِ، أَنُؤَاخَذُ بِمَا نَقُولُ؟ قَالَ: «ثَكِلَتْكَ أُمُّكَ يَا ابْنَ جَبَلٍ، وَهَلْ يَكُبُّ النَّاسَ فِي النَّارِ عَلَى مَنَاخِرِهِمْ إِلَّا حَصَائِدُ أَلْسِنَتِهِمْ؟» قَالَ حَبِيبٌ فِي هَذَا الْحَدِيثِ: «وَهَلْ تَقُولُ شَيْئًا إِلَّا لَكَ أَوْ عَلَيْكَ»



    7 - حَدَّثَنِي حَمْزَةُ بْنُ الْعَبَّاسِ، أَخْبَرَنَا عَبْدَانُ بْنُ عُثْمَانَ، أَخْبَرَنَا عَبْدُ اللَّهِ، أَنَا مَعْمَرٌ، عَنِ الزُّهْرِيِّ، عَنْ عَبْدِ الرَّحْمَنِ بْنِ مَاعِزٍ، عَنْ سُفْيَانَ بْنِ عَبْدِ اللَّهِ الثَّقَفِيِّ قَالَ: قُلْتُ: يَا رَسُولَ اللَّهِ، حَدِّثْنِي بِأَمْرٍ أَعْتَصِمُ بِهِ. قَالَ: " قُلْ: رَبِّيَ اللَّهُ ثُمَّ اسْتَقِمْ ". قَالَ: قُلْتُ: يَا رَسُولَ اللَّهِ، مَا أَخْوَفُ مَا تَخَافُ عَلَيَّ؟ فَأَخَذَ بِلِسَانِهِ ثُمَّ قَالَ: «هَذَا»

    The whole book has the asterisks separators and the numbers of page, chapter and hadiths deleted in no times not even 2 seconds.

    I ❤ GNU/Linux, FOSS and CLI

  • Level Up Your Bash Scripts: $?, Custom Exit Codes & Fail-Safe Flags

    In our previous post about exit code, we learned why exit 0 and exit 1 are the difference between a script that “smiles through a disaster” and one that actually warns you. But manually writing exit 1 after every command quickly becomes tedious, especially as your scripts grow. How do production-grade scripts handle errors without cluttering every line with checks?

    The answer lies in three Bash superpowers: the $? variable, standardized exit codes, and the set -euo pipefail safety trio. Let’s break them down with real examples.

    By default, Bash is extremely optimistic—it assumes errors are optional, missing variables are fine, and partial success counts as success. While this behavior feels charming in a local terminal, it becomes reckless in automation. Out of the box, Bash will happily execute a command that fails, ignore the failure, and continue as if nothing happened. That’s not resilience; it’s denial. If your script controls anything real, silent failure is the worst possible outcome. The truly dangerous scripts aren’t the ones that crash—they’re the ones that fail halfway, leave systems in weird states, and still exit with code 0. Those scripts pass CI, get promoted, and quietly break production later. Congratulations—you’ve automated uncertainty.


    1. The Hidden Power of $?

    Every command you run in Bash leaves behind a receipt. That receipt is the $? variable. It holds the exit status of the most recently executed command. The catch? It updates after every command, including echo or variable assignments. If you don’t capture it immediately, it’s overwritten and gone forever.

    Watch what happens in this broken example:

    #!/bin/bash
    ls /nonexistent_dir 2>/dev/null
    echo "Exit code was: $?"   # ✅ Prints 2 (directory not found)
    
    echo "Just checking..."    # ⚠️ This runs successfully
    echo "Now $? is: $?"       # ❌ Prints 0! The error receipt was replaced.

    The Fix: Capture Before It Changes
    Always store $? in a named variable the moment you need it:

    #!/bin/bash
    rsync -av /data/ /backup/ 2>/dev/null
    result=$?  # ✅ Save the receipt immediately
    
    if [ $result -ne 0 ]; then
        echo "Rsync failed with exit code: $result"
        exit $result
    fi

    Pro Tip: You can often skip $? entirely. Bash checks exit codes natively in if statements:

    if ! cp critical_config.yml /etc/app/; then
        echo "Config copy failed!"
        exit 1
    fi

    The ! negates the success, so the if block only runs when cp returns non-zero. Cleaner, safer, and less prone to $? overwrites.


    2. Beyond exit 0 & exit 1: Custom Exit Codes

    exit 0 means success. exit 1 means generic failure. But what if your script can fail for five different reasons? Telling a monitoring system or CI/CD pipeline “it failed” isn’t enough. Bash supports exit codes 0–255, and the Linux ecosystem follows established conventions:

    • 0: Success
    • 1: General/catchall error
    • 2: Misuse of shell builtins (wrong flags/arguments)
    • 126: Command found but not executable
    • 127: Command not found
    • 128+N: Script killed by signal N (e.g., 130 = Ctrl+C)

    You’re free to define your own codes for application-specific errors. Just pick numbers that won’t collide with signals (usually 10–99 works well):

    #!/bin/bash
    set -euo pipefail
    
    # Check source directory
    if [ ! -d "/var/app/data" ]; then
        echo "ERROR: Source directory missing"
        exit 10  # Custom: Configuration error
    fi
    
    # Check disk space
    usage=$(df /var/app/data | awk 'NR==2 {print $5}' | tr -d '%')
    if [ "$usage" -gt 90 ]; then
        echo "ERROR: Disk usage at ${usage}%. Backup aborted."
        exit 20  # Custom: Resource constraint
    fi
    
    echo "✅ Backup completed successfully."
    exit 0

    Now, an automation tool reading the exit code knows exactly what went wrong: 10 means fix the config path, 20 means clear disk space. Always document your custom codes in a header comment so your team (or future you) doesn’t have to guess.


    3. The “Fail-Safe” Script: set -euo pipefail

    Manually checking every exit code works for tiny scripts. For anything running on a schedule or in production, it’s fragile. Bash provides built-in safety switches that act as an automatic error net. Place these at the top of your script:

    #!/bin/bash
    set -euo pipefail

    Here’s what each flag does:

    • set -e (errexit): Exits immediately if any command returns non-zero. No more silent failures cascading into data loss.
    • set -u (nounset): Treats unset variables as errors. Prevents typos like $backp_dir from accidentally creating empty folders.
    • set -o pipefail: Changes how pipelines behave. By default, cmd1 | cmd2 | cmd3 only returns the exit code of cmd3. With pipefail, it returns the last non-zero exit code in the chain.

    See the difference in action:

    # Default Bash behavior
    false | echo "Pipe succeeded"
    echo $?  # Prints 0 (echo's success hid false's failure)
    
    # With set -o pipefail enabled
    false | echo "Pipe succeeded"
    echo $?  # Prints 1 (Bash caught the hidden failure)

    What if you expect a command to fail sometimes? You can temporarily disable -e:

    set +e  # Turn off errexit
    ping -c 1 192.168.1.50 || echo "Host unreachable"
    set -e  # Turn protection back on immediately

    🔍 The “Truth Test” for All Three

    Run this in your terminal to see how they work together:

    bash -c 'set -euo pipefail; false | grep "test"; echo "This never prints"'
    echo "Exit: $?"

    Result: The script stops at false | grep, returns 1, and the final echo never runs. The computer sees the truth.

    📝 Summary

    • $? is a temporary receipt. Capture it instantly or let Bash handle it in if statements.
    • Custom exit codes turn “it broke” into actionable diagnostics for humans and machines.
    • set -euo pipefail automates error catching, so you don’t have to manually guard every line.

    Start adding these to your scripts today. Your backups, deployments, and sanity will thank you.

  • Upgrading my rsync script

    Upgrading my rsync script

    A Basic Improvement in My rsync Script

    rsync is used for efficient file synchronization and backup. It copies only the differences between source and destination, saving time and bandwidth.

    And it is preinstalled on Ubuntu.

    I use rsync to back up some of my folders to external storage like this:

    
    
    
    
    

    As you can see, I use && and \.

    • && is used to combine two bash commands and run the second command only if the first command succeeds (exits with status 0).
    • \ is used for line continuation – it tells the shell that the command continues on the next line, making long commands more readable.

    We could use ; instead of &&, but ; would run the second command regardless of whether the first command succeeded or failed. && is safer for backup operations because if the first backup fails, the second won’t run, preventing incomplete or corrupted backups.


    The Improved Script with a Loop

    The problem with my original approach is that I had to manually write a line for each external drive. If I added a new drive, I had to update the script. Also, if a drive wasn’t connected, rsync would still try to run and throw an error.

    Here’s my improved script that automatically handles multiple drives and checks if they’re connected:


    Array definition. Creates an array variable named DRIVES containing four paths (one per drive). The parentheses () define an array, and each quoted string is an element. Using an array allows us to loop through all drives without repeating code.

    The Very Important Symbol: [@]

    The critically important symbol is [@] (at-sign with brackets).


    Why [@] is So Important

    What it does:

    ${DRIVES[@]} expands to all elements of the array DRIVES, with each element treated as a separate word.

    The Danger of NOT using [@]

    If you wrote this incorrectly as:

    for DRIVE in ${DRIVES[@]}; do   # Missing quotes - WRONG!

    Or worse:

    for DRIVE in $DRIVES; do        # Just wrong - treats array as single string

    Here’s what happens with a drive path containing spaces, like "/media/fakhri/SAMSUNG SSD":

    Correct WayIncorrect Way
    "${DRIVES[@]}"${DRIVES[@]} (no quotes)
    SAMSUNG SSD stays as ONE itemSAMSUNG and SSD become TWO separate items
    The script sees: /media/fakhri/SAMSUNG SSDThe script sees:
    1. /media/fakhri/SAMSUNG
    2. SSD (which is not a valid path)

    The Three Array Expansion Options Compared

    SymbolBehaviorWhen to Use
    $DRIVESOnly first element (treats array as scalar)Never for arrays
    ${DRIVES[*]}All elements as single stringWhen you want one combined string
    ${DRIVES[@]}All elements as separate wordsMost common – use in loops
    "${DRIVES[@]}"All elements as separate words, preserving spacesALWAYS USE THIS for paths with spaces

    The Golden Rule

    Always use "${ARRAY[@]}" with quotes when iterating over arrays containing file paths.

    The quotes + [@] combination ensures:

    1. Each array element stays intact (spaces preserved)
    2. Empty elements are preserved
    3. Special characters (like * or ?) are not expanded

    Without this, your backup script will fail silently and try to write to completely wrong locations!

    Every Important Symbol Explained

    Here’s the breakdown of every critical symbol in these lines:

    if [ -d "$DRIVE" ]; then
        # ... backup commands ...
    else
        echo "Skipping $DRIVE (Drive not connected)"
    fi
    done

    if [ -d "$DRIVE" ]; then

    SymbolNameWhat it does
    ifKeywordStarts a conditional statement. If the following command returns true (exit code 0), execute the code between then and else/fi
    [Test command (left bracket)A built-in command that evaluates conditional expressions. Must have spaces around it! [ -d "$DRIVE" ] not [-d "$DRIVE"]
    (space)Space separatorRequired between [ and -d – bash needs spaces to distinguish commands from arguments
    -dFlag (directory test)Tests if the following path exists and is a directory. Returns true (0) if yes, false (1) if not
    (space)Space separatorRequired between -d and the path
    "$DRIVE"Double-quoted variableExpands to the value of DRIVE variable while preserving spaces in the path. Without quotes, a path like /media/fakhri/SAMSUNG SSD would break into two words
    (space)Space separatorRequired between the path and the closing bracket
    ]Closing bracketEnds the test command. Must have a space before it!
    ;Command separatorAllows multiple commands on one line. Here it separates the test command from then
    thenKeywordMarks the beginning of the code block to execute if the if condition is true

    else

    SymbolNameWhat it does
    elseKeywordMarks the alternative code block. Executes if the if condition was false (the drive was NOT a directory)

    echo "Skipping $DRIVE (Drive not connected)"

    SymbolNameWhat it does
    echoCommandPrints text to the terminal
    " "Double quotesEverything inside becomes a single argument to echo, even if it contains spaces or variables. Variables inside ($DRIVE) still expand
    $DRIVEVariable expansionReplaces $DRIVE with its actual value (e.g., /media/fakhri/32Go)
    ()ParenthesesRegular text characters here – just part of the message. Not a command substitution because there’s no $ before them

    fi

    SymbolNameWhat it does
    fiKeywordCloses the if block. It’s “if” spelled backwards. Every if must have a matching fi

    done

    SymbolNameWhat it does
    doneKeywordCloses the for loop. Marks the end of the loop body. Every for must have a matching done

    The Most Critical Symbol: [ ] (Test Command)

    The brackets [ ] are NOT syntax – they are a command!

    Mental model:

    if [ -d "$DRIVE" ]; then

    Is equivalent to:

    if test -d "$DRIVE"; then

    The [ command is just an alias for test that requires a closing ].

    Common Mistakes with [ ]:

    ❌ Wrong✅ CorrectWhy
    [-d "$DRIVE"][ -d "$DRIVE" ]Missing spaces – bash can’t find the [ command
    [$DRIVE][ -n "$DRIVE" ]No flag – what are you testing?
    [ -d $DRIVE ][ -d "$DRIVE" ]No quotes – path with spaces breaks

    Symbol Hierarchy in Context

    if [ -d "$DRIVE" ]; then
    │  │ │ │        │  │
    │  │ │ │        │  └── ends the "then" block start
    │  │ │ │        └── separates test from "then"
    │  │ │ └── variable expands to actual path
    │  │ └── tests if path is a directory
    │  └── starts test command
    └── begins conditional
    
    then
    │
    └── marks true block
    
    else
    │
    └── marks false block
    
    echo "Skipping $DRIVE (Drive not connected)"
    │    │                    │
    │    │                    └── variable inside quotes expands
    │    └── quotes keep everything as one argument
    └── prints to terminal
    
    fi
    │
    └── closes if
    
    done
    │
    └── closes for loop

    Quick Reference Card

    SymbolMeaningRemember by
    ifBegin conditional“if this is true…”
    [Start testLeft bracket opens the test
    -dDirectory check“-d” for “directory”
    $Variable valueDollar = value
    " "Preserve spacesQuotes = togetherness
    ]End testRight bracket closes the test
    ;Command separatorSemicolon = stop then go
    thenTrue branch“then do this…”
    elseFalse branch“otherwise do this…”
    fiEnd if“if” backwards
    doneEnd loop“for…done”

    The most important takeaway: [ ] needs spaces inside and outside, and always quote your variables inside tests!

    Final Note: Use [[ ]] Instead of [ ]

    For Bash scripts, the double-bracket [[ ]] is superior to the single-bracket [ ] used in this article. Unlike [ ] (a command that requires spaces and quoted variables), [[ ]] is a Bash keyword that prevents word splitting and pathname expansion. This means you can write [[ -d $DRIVE ]] without quotes, even if the path contains spaces. [[ ]] also supports pattern matching (== *.txt), regex matching (=~), and natural logical operators (&&, ||). Stick with [ ] only if you need portability to other shells like sh; otherwise, always prefer [[ ]] for cleaner, safer, and more readable conditional tests.

  • Don’t Let Your Script Lie to You: A Guide to Exit Status

    Don’t Let Your Script Lie to You: A Guide to Exit Status

    To see why exit 1 and exit 0 are so important, we have to look at what happens when a script lies to the computer.

    If you omit them, Bash simply reports the exit status of the very last command that ran. This can lead to a “False Success.”

    1. The “Broken” Script (No Exit Codes)

    Save this as broken_backup.sh. Notice there is no exit 1.

    Bash


    2. The “Good” Script (With Exit Codes)

    Save this as good_backup.sh.

    Bash

    3. How to Test Them (The “Truth” Test)

    Run these commands in your terminal one after the other. We will use the && operator, which only runs the second command if the first one reports Success (0).

    Testing the Broken Script:

    Bash

    Result: Even though the script printed “ERROR,” the computer saw the final echo succeeded, so it ran the “SUCCEEDED” message. This is dangerous because a backup could fail and you wouldn’t know!

    Testing the Good Script:

    Bash

    Result: The script stops at exit 1. The computer sees the 1, skips the && part, and triggers the || (failure) part.

    Here are the results as copied from my terminal:

    Why this matters in the real world

    Imagine you have a script that:

    1. Deletes your old files.
    2. Copies your new files (The Backup).
    3. Cleans up the temporary folder.

    If Step 2 (The Backup) fails because the disk is full, but you didn’t write exit 1, the script will continue to Step 3 and delete your only remaining copies, thinking everything is fine!

    Summary: * With exit 1: The script “screams” when there is a problem.

    • Without it: The script “whispers” an error but smiles at the computer, pretending everything is perfect.

    Note: about the use of “bash” at the start of the command below:


    1. Manual Interpreter: Typing bash before your filename manually tells the system to use the Bash program to translate and run your script’s code.

    2. Bypasses Permissions: It allows you to execute a script immediately without needing to set “executable” permissions via chmod +x.

    3. Ensures Compatibility: It guarantees the script runs in the full Bash environment rather than a limited shell (like sh) that might misunderstand your syntax.

    4. Testing Logic: It is the most reliable way to test if your exit 0 and exit 1 codes are working correctly before finalizing the script for automation.

  • Compress and archive files/directories in Gnu/Linux OS

    Compress and archive files/directories in Gnu/Linux OS


    Use Gzip to compress


    The standard zipping/compression program in Gnu/Linux OS is gzip that is Gnu Zip. The extension of Gzip is “.gz” or “.z

    To compress a single file only using gzip just:

    $ gzip filename


    To compress many files do as shown below:

    (more…)