SIDEBAR
»
S
I
D
E
B
A
R
«
Syntax highlighting in pagers (eg. like less) using GNU source-highlight
Mar 21st, 2021 by miki

EDIT 2021-08-31: add section about possible unexpected change of behaviour when using the simple (global) setup (fx. git)

There is this nifty OS project, GNU, which has this nifty piece of software for syntax highlighting, source-highlight (aka. src-highlite), together with which is distributed this nifty shell script, src-hilite-lesspipe.sh, meant for piping arbitrary text through the highlighter selecting and applying a sensible highlight language definition, before being paged in some nifty pager which is able to interpret ANSI escape codes (ISO/IEC 6429 or ECMA-48, previously ANSI X3.64/FIPS PUB 86) like fx. the nifty and ubiquitous less.

Sadly, even after having installed less and source-highlight on modern Ubuntu and Debian systems they are not inter-operating by default. You’d have to feel the itch of syntax highlighting, discover source-highlight and dig its documentation to find out about said script.

TL;DR – quick and simple setup

Below is a quick two-line shell HOW-TO which sets up environment variables for the current user to enable auto-detection of language and subsequent syntax highlighting pr. default in less using GNU source-highlight (here done on Ubuntu 20.04 LTS, should behave similarly on Debian and derived distributions);

$ sudo apt install source-highlight
$ echo -e "\nexport LESSOPEN=\"| /usr/share/source-highlight/src-hilite-lesspipe.sh %s\"\nexport LESS=' -R '" >> ~/.bash_aliases

 

This setup will make fx. some C source code display as below in less.

environment set up for less to enable ANSI color codes and pass any text through source-highlighter for potential ANSI escape code addition

less showing highlighted C code, auto-detected and highlighted by GNU source-highlighter

Complicating TL;DR

One caveat of this global setup being active for any invocation of less, is that most programs will behave accordingly and some maybe different from expectations.

Notably, using the git command line client with the above setup will make git refrain from setting a default LESS=FRX environment (see core.pager of man git config) when invoking less. This turns off the less features “quit-if-one-screen” (F) and “no-init” (X) (see OPTIONS of man less) which usually makes less invisible in git contexts unless paging is actually needed. This could cause confusion for some, making git seemingly take over the terminal on even 1 line outputs (see this bug report on GNU src-hilite and my comment detailing the above).

One solution to this would of course be to add “FX” to the environment, another is making a little more elaborate, more conservative and less transparent setup, as below.

For the conservative

If you’d still like to have a “plain less” not messing with and amending you text unless you ask it to, you could make an alias to use specifically when you want syntax highlighting. Put these somewhere interpreted by you shell (for bash fx. ~/.bash_aliases);

# syntax highlight in less
alias lesssh="LESSOPEN='|/usr/share/source-highlight/src-hilite-lesspipe.sh %s' LESS=' -R ' less"
function lessurl() { wget -O- -q $1 |source-highlight -f esc -s html |less -R; }

Line two is a bonus shell function pulling some HTML from a webserver using wget, adding syntax highlighting and showing it in less.

Digging deeper

For the brave, take a tour of the “info source-highlight” (or “man source-highlight”) manuals (also here) to become familiar with the tool. You can use it anywhere you’d like some colour on arbitrary text and where color are supported in various ways and encodings, for example HTML and latex;

source-highlight adding color encoding to C code using HTML and latex encoding

If you are a programmer wanting to add highlighting features to you own application, the command line utilities are building on a highlighting library which you can utilise (API documentation here).

Automating wav to ogg conversion
Jun 6th, 2019 by miki

EDIT 2021-07-11: the script described below has been added to the author’s hometools repository on sourcehut.org

Needed some space on the disk.

Had some music, some of it were wav files.

Threw together this script to transmogify it into ogg using ffmpeg.

Statistics say that 24 wav files occupying 1,7 GiB was made into 24 ogg files occupying 105 MiB, a reduction to ~6% (du -h calculates 2^10 numbers but uses SI 10^3 prefixes..).

miki@filly:~/Music$ find . -iname "*.wav" -print0 | sed s/.wav/.ogg/g |du --files0-from=- -h -c|tail -1
105M	total
miki@filly:~/Music$ find . -iname "*.wav" -print0 | sed s/.wav/.wav/g |du --files0-from=- -h -c|tail -1
1,7G total
miki@filly:~/Music$ find . -iname "*.wav" -print0 | sed s/.wav/.ogg/g |du --files0-from=- -h |wc -l
24
miki@filly:~/Music$

Script

Install prerequisite ffmpeg and ffprobe utilities on apt distributions by doing “apt install ffmpeg”.

BEWARE: script deletes any wav files in current directory and below without warning if conversion to ogg is fine according to ffmpeg return code and stdout/err.

Manual confirmation of all deletions can be enabled by uncommenting line 8 (CONFIRM=..).

It could (very) easily be amended for conversion between any container or compression format supported by ffmpeg by replacing “wav” and “ogg” in glob pattern and OGG variable definition (for completeness also in echo output…).

miki@filly:~/wavtest$ cat ~/hometools/wav2ogg.sh
#! /bin/bash
#
# Make .ogg out of all .wav in cwd and below, remove .wav!
#
# When converting remove only if conversion is ok.
# ffprobe any present .ogg and check if playtime is same as wav, remove only if all is good.

#CONFIRM=-i # confirm deletion?

REQS="ffmpeg ffprobe"
which $REQS  >/dev/null
if [ ! $? -eq 0 ]; then
    echo We need $REQS to do stuff!
    exit 1
fi

shopt -s globstar nullglob nocaseglob # enable 2-star magic
for f in **/*.wav; do
    OGG=$(dirname "$f")/$(basename "$f" wav)ogg
    echo -n "$f: "
    if [ ! -f "$OGG" ]; then
        echo -en "\e[33mno ogg\e[0m"
        CONVERT=1
    else
        echo -en "\e[32malready there\e[0m"
        if [ ! "$(ffprobe "$f" 2>&1|grep Duration|cut -d, -f1)" = "$(ffprobe "$OGG" 2>&1|grep Duration|cut -d, -f1)" ]; then
            echo -en " - \e[31mogg corrupt\e[0m"
            CONVERT=1
            rm $CONFIRM "$OGG"
        else
            OGGOK=1
        fi
    fi
    if [ $CONVERT ]; then
        echo -en " - \e[33mconverting\e[0m"
        OUT=$(ffmpeg -loglevel error -i "$f" "$OGG" 2>&1)
        if [ ! $? -eq 0 ]; then
            echo -e " \e[31mfatal error!\e[0m\nffmpeg output was:\n\n$OUT\n"
            rm $CONFIRM "$OGG"
        elif [ ! -z "$OUT" ]; then
            echo -e " \e[31mnon-fatal error!\e[0m\nffmpeg output was:\n\n$OUT\n"
        else
            OGGOK=1
        fi
    fi
    if [ $OGGOK ]; then
        echo -e " - all good removing wav"
        rm $CONFIRM "$f"
    fi
done
miki@filly:~/wavtest$ 

Usage and Output

Before

miki@filly:~/wavtest$ ls -lR
.:
total 59964
drwxrwxr-x 3 miki miki     4096 Jun  6 22:33 one
-rw-rw-r-- 1 miki miki 61390864 Jun  6 22:32 Sordid Affair (Man Without Country Version).wav

./one:
total 30720
drwxrwxr-x 3 miki miki     4096 Jun  6 22:34 two
-rw-rw-r-- 1 miki miki 31451304 Jun  6 22:30 Vision Man MASTER.wav

./one/two:
total 55352
-rw-rw-r-- 1 miki miki 56675660 Jun  6 22:34 Bye Barat - Going For Broke (UNKWON´s Autumn Remix) LIM_02.wav
drwxrwxr-x 2 miki miki     4096 Jun  6 22:34 three

./one/two/three:
total 375112
-rw-rw-r-- 1 miki miki 384108056 Jun  6 22:30 Rigoremix - Pornographe - Final2.wav
miki@filly:~/wavtest$ du -hs .
509M	.
miki@filly:~/wavtest$

During

In real life, also in technicolor!

miki@filly:~/wavtest$ ~/hometools/wav2ogg.sh 
one/two/Bye Barat - Going For Broke (UNKWON´s Autumn Remix) LIM_02.wav: no ogg - converting - all good removing wav
one/two/three/Rigoremix - Pornographe - Final2.wav: no ogg - converting - all good removing wav
one/Vision Man MASTER.wav: no ogg - converting - all good removing wav
Sordid Affair (Man Without Country Version).wav: no ogg - converting - all good removing wav
miki@filly:~/wavtest$

After

miki@filly:~/wavtest$ ls -lR
.:
total 4952
drwxrwxr-x 3 miki miki    4096 Jun  6 22:35 one
-rw-rw-r-- 1 miki miki 5066460 Jun  6 22:35 Sordid Affair (Man Without Country Version).ogg

./one:
total 2528
drwxrwxr-x 3 miki miki    4096 Jun  6 22:34 two
-rw-rw-r-- 1 miki miki 2580752 Jun  6 22:35 Vision Man MASTER.ogg

./one/two:
total 4680
-rw-rw-r-- 1 miki miki 4785594 Jun  6 22:34 Bye Barat - Going For Broke (UNKWON´s Autumn Remix) LIM_02.ogg
drwxrwxr-x 2 miki miki    4096 Jun  6 22:35 three

./one/two/three:
total 19456
-rw-rw-r-- 1 miki miki 19922936 Jun  6 22:35 Rigoremix - Pornographe - Final2.ogg
miki@filly:~/wavtest$ du -hs .
31M .
miki@filly:~/wavtest$

Metadata

ffmpeg does some nice metadata conversions out of the box;

miki@filly:~/Music$ ffprobe ~/"wavbak/Delayscape  A Short While Rigolin Edit Audaccity.wav" 2>&1| grep Input -A 1000
Input #0, wav, from '/home/miki/wavbak/Delayscape  A Short While Rigolin Edit Audaccity.wav':
  Metadata:
    title           : Delayscape - A Short While (Rigolin Edit)
    artist          : rigolin
    comment         : Love Melody
    date            : 2018
  Duration: 00:06:33.97, bitrate: 3072 kb/s
    Stream #0:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, 2 channels, flt, 3072 kb/s
miki@filly:~/Music$ ffprobe "./Delayscape - Heinz Beauvaix/Soundcloud/Delayscape  A Short While Rigolin Edit Audaccity.ogg" 2>&1| grep Input -A 1000
Input #0, ogg, from './Delayscape - Heinz Beauvaix/Soundcloud/Delayscape  A Short While Rigolin Edit Audaccity.ogg':
  Duration: 00:06:33.97, start: 0.000000, bitrate: 99 kb/s
    Stream #0:0: Audio: vorbis, 48000 Hz, stereo, fltp, 112 kb/s
    Metadata:
      ENCODER         : Lavc56.60.100 libvorbis
      TITLE           : Delayscape - A Short While (Rigolin Edit)
      ARTIST          : rigolin
      comment         : Love Melody
      DATE            : 2018
miki@filly:~/Music$

Source Material

»  Substance:WordPress   »  Style:Ahren Ahimsa
© 2021 Mikkel Kirkgaard Nielsen, contents CC BY-SA 4.0