Advanced Bash regex with examples
Regular expressions (regex or regexp) in Bash are powerful tools for pattern matching and text manipulation. They allow you to define search patterns and perform complex matching operations on strings. Here's an explanation of some advanced regex concepts and examples of how to use them in Bash:
Understanding Regular Expressions (regex)
- Regex are powerful tools for pattern matching in text, enabling you to find, extract, or replace specific patterns based on complex rules.
- They use a special syntax to define these rules, combining literal characters, metacharacters, and quantifiers.
^: Anchors the regex at the beginning of the line.
$: Anchors the regex at the end of the line.
.: Matches any single character.
*: Matches zero or more occurrences of the preceding character or group.
+: Matches one or more occurrences of the preceding character or group.
?: Matches zero or one occurrence of the preceding character or group.
[]: Matches any single character within the brackets.
Example:
# Check if a line starts with "Hello"
if [[ $line =~ ^Hello ]]; then
echo "Line starts with Hello"
fi
# Check if a line ends with "world"
if [[ $line =~ world$ ]]; then
echo "Line ends with world"
fi
Character Classes
\d: Matches any digit (equivalent to [0-9]).
\D: Matches any non-digit.
\w: Matches any word character (alphanumeric + underscore).
\W: Matches any non-word character.
\s: Matches any whitespace character.
\S: Matches any non-whitespace character.
Example:
# Check if a line contains a phone number
if [[ $line =~ \b\d{3}-\d{3}-\d{4}\b ]]; then
echo "Line contains a phone number"
fi
Quantifiers
{n}: Matches exactly n occurrences of the preceding character or group.
{n,}: Matches n or more occurrences of the preceding character or group.
{n,m}: Matches between n and m occurrences of the preceding character or group.
Example:
# Check if a line contains 3 or more consecutive digits
if [[ $line =~ [0-9]{3,} ]]; then
echo "Line contains 3 or more consecutive digits"
fi
Grouping and Capturing
(): Groups patterns together.
|: Acts as a logical OR between patterns.
\1, \2, etc.: Refers to captured groups.
Example:
# Check if a line contains either "apple" or "orange"
if [[ $line =~ (appleorange) ]]; then
echo "Line contains apple or orange"
fi
Anchors and Word Boundaries
\b: Matches a word boundary.
Example:
# Check if a line contains the word "error" as a whole word
if [[ $line =~ \berror\b ]]; then
echo "Line contains the word 'error'"
fi
Advanced Features
Character Classes
[:alnum:]: Matches alphanumeric characters (letters and numbers)
if [[ $input =~ [[:alnum:]] ]]; then
echo "Input contains at least one alphanumeric character"
fi
[:alpha:]: Matches alphabetic characters (letters)
if [[ $input =~ [[:alpha:]] ]]; then
echo "Input contains at least one alphabetic character"
fi
[:digit:]: Matches digit characters (numbers)
if [[ $input =~ [[:digit:]] ]]; then
echo "Input contains at least one digit"
fi
[:space:]: Matches whitespace characters (spaces, tabs, newlines)
if [[ $input =~ [[:space:]] ]]; then
echo "Input contains at least one whitespace character"
fi
[:punct:]: Matches punctuation characters
if [[ $input =~ [[:punct:]] ]]; then
echo "Input contains at least one punctuation character"
fi
Backreferences
Backreferences (\1, \2, etc.): Refer to previously matched subexpressions within parentheses:
if [[ $line =~ (foo)bar ]]; then
echo "Line contains 'foobar'"
echo "The value of the captured group is: ${BASH_REMATCH[1]}"
fi
Positive Lookbehind (?<=pattern): Matches only if the preceding pattern exists but is not captured:
if [[ $line =~ (?<=prefix)word ]]; then
echo "Line contains 'word' preceded by 'prefix'"
fi
Negative Lookbehind (?<!pattern): Matches only if the preceding pattern does not exist:
if [[ $line =~ (?<!not)allowed ]]; then
echo "Line contains 'allowed' not preceded by 'not'"
fi
Positive Lookahead (?=pattern): Matches only if the following pattern exists but is not captured:
if [[ $line =~ word(?=suffix) ]]; then
echo "Line contains 'word' followed by 'suffix'"
fi
Negative Lookahead (?!=pattern): Matches only if the following pattern does not exist:
if [[ $line =~ word(?!=forbidden) ]]; then
echo "Line contains 'word' not followed by 'forbidden'"
fi
Note: Bash's regex support is based on the extended regular expression (ERE) syntax. Lookbehind assertions are not directly supported in Bash. The examples with lookbehind are just for illustration and may not work as expected in Bash. The positive and negative lookahead examples should work fine. If you need more advanced lookbehind support, you might consider using other tools like grep -P (Perl-compatible regex) or external tools like awk or sed.
Non-Capturing Groups
(?:pattern): Capture the pattern for matching but not for referencing.The (?:pattern) syntax is used for non-capturing groups in regular expressions. This means that the group is used for matching, but the matched content is not stored for later reference using backreferences.
if [[ $line =~ (foo(?:bar)) ]]; then
echo "Line contains 'foobar'"
echo "The value of the non-capturing group is: ${BASH_REMATCH[1]}"
else
echo "Line does not contain 'foobar'"
fi
In this example, the non-capturing group (?:bar) is used to match the string "bar" after "foo". The entire pattern (foo(?:bar)) is capturing only "foobar," and the value of the non-capturing group (?:bar) is not stored in ${BASH_REMATCH[1]}.
Bash Specifics
- Use the =~ operator for regex matching (e.g., if [[ $string =~ pattern ]]; then ...).
- Escape reserved characters in Bash like $ and [] with \ (e.g., \$$).
- Consider alternative tools like awk, sed, or Perl for more complex regex work.
Conclusion
Regular expressions in Bash enable advanced pattern matching for text manipulation. Through constructs like anchoring, character classes, quantifiers, and lookarounds, users can define intricate search patterns for matching and manipulating strings in shell scripts, enhancing the power of text processing operations.
Related Topics