Below you’ll find an illustrated guide for freeing the authentication mechanism used by the Google Authenticator app for Android or iPhone for use on your favorite device (anywhere an implementation is available). In fact the Authenticator is using a standards based 2FA (two factor authentication) scheme defined by OATH (Initiative for Open Authentication) and published in RFC6238 dubbed TOTP – Time-Based One-Time Password Algorithm (more Authenticator background and it’s basis HOTP).

This is a rather fine contraption but Google doesn’t advertise it very loudly being a standard instead locking the generated TOTP secret into a QR code that they will only imply are for use by their own Google Authenticator. That is not true as a host of alternative Android OTP apps are compatible and can read the QR codes as they are based upon the Authenticator’s legacy as an open source application which Google took private.

Retrieve Google Account QR Code

Login to your Google Account (maybe using the authenticator?) and go to Account -> Security -> Signin -> 2-step verification.

Locate the “Authenticator app” section and Click “CHANGE PHONE” (really “CHANGE TOTP SECRET”).

In my country’s locale, Danish, this is BTW mistranslated as “SKIFT TELEFONNUMMER” = change phone number);

You’ll now get a choice between either iPhone or Android, this will only affect the link to the app store shown on next screen which also contains the QR code, the one we are really after:

Right click the QR code image, select to save it and put it somewhere you can find it (~/Google_TOTP_QR.png might be sensible).

Get your tools right

Install the QR-code decoder zbar for extracting the TOTP secet from the image and the OATH toolkit oathtool for generating future TOTP passwords using it. On Debian/Ubuntu this can be done by installing the packages zbar-tools and oathtool.

$ sudo apt install zbar-tools oathtool

Now extract the otpauth URI (seems to be a Google thing) by passing the image file to zbarimg;

$ zbarimg ~/Google_TOTP_QR.png
 QR-Code:otpauth://totp/Google%3Auser%40domain.com?secret=pca7uyfht7f6mfs7oiec4aeavxaevish&issuer=Google
 scanned 1 barcode symbols from 1 images in 0.02 seconds

The URI contains all parameters to input into the TOTP algorithm for generating a password usable for 2FA authentication, notably the secret key in base32 format. Defaults are not mentioned in the URI and are not necessary to specify explicitly for oathtool. So for the above URI specifying only a secret, a password can be generated as such;

$ oathtool --totp --base32 pca7uyfht7f6mfs7oiec4aeavxaevish
 491293

“491293” being the password, having a default validity of 30 seconds (called step size because of the counter based nature of the HOTP behind it).

If your are interested in the defaults assumed by oathtool they can be viewed by supplying -v to it (not showing TOTP mode=SHA512, I later added this feature in my oath-toolkit fork, and opened a MR) ;

$ oathtool --totp --base32 pca7uyfht7f6mfs7oiec4aeavxaevish -v
Hex secret: 7881fa60a79fcbe6165f72082e0080adc04aa247
Base32 secret: PCA7UYFHT7F6MFS7OIEC4AEAVXAEVISH
Digits: 6
Window size: 0
Step size (seconds): 30
Start time: 1970-01-01 00:00:00 UTC (0)
Current time: 2017-11-06 23:23:31 UTC (1510010611)
Counter: 0x30007F7 (50333687)

133339

Use & Behold

Enjoy having cut your reliance on the Google Autenticator, and opened up for your own choice of client:

• Browser: https://daplie.github.io/browser-authenticator/
• Android:
  1. AndOTP: https://f-droid.org/packages/org.shadowice.flocke.andotp/
  2. FreeOTP:
  3. OTP Authenticator: https://f-droid.org/packages/net.bierbaumer.otp_authenticator/
• Node:
  1. https://github.com/guyht/notp
  2. https://github.com/yeojz/otplib
  3. https://github.com/njl07/otp.js
• PHP:
• Hardware:
  1. Token 2: https://token2.com/?content=hwtokens
  2. NitroKey Pro: https://shop.nitrokey.com/shop/product/nitrokey-pro-3

Also see the dongleauth.info website for a list of other sites supporting TOTP and U2F.

Clean Up & Secure

Be sure to delete the QR code image when finished, and seek out a suitably secure place to store your TOTP secret. One solution could be using PGP encryption, maybe even through some clever management system such as pass.

Also educate yourself of the shortcomings of TOTP/HOTP and consider studying and using the FIDO U2F standard and related devices.

Got myself an exported message from (apparently) Exchange/Outlook in a file with .msg extension. My initial thought was that this was just a plain ASCII email (seem to remember having handled .msg files as such at some point), but looking at it as text exposed a load of binary and the “file” utility reported it being of type “Composite Document File V2 Document, Cannot read section info”.

What is it?

Some searching reveals .msg is in a proprietary, but largely documented, format called “Outlook Item (.msg) File Format” (or formally, MS-OXMSG, find the specification  in the MSDN document entitled “[MS-OXMSG]: Outlook Item (.msg) File Format“). This format is related to the CFB/CFBF ([MS-CFB]: Compound File Binary File Format).

Check out what Library of Congress’ nice Sustainability of Digital Formats project has to say about this format in their evalutaion of its sustainability (fx. that Microsoft does not advise to use it for sending information to an unknown receiver, Microsoft source).

Extension

Interpreting it

The availability of open tools for parsing this sort of thing, especially when distributing it verbatimly, should of course be a consideration before doing such. Luckily some options are available however they seem not to be widely deployed on default installs in the open computing world, so some mocking about is necessary.

libgsf

The GNOME project has developed an open source library and some tools for interacting with files of this type, check out libgsf documentation at developer.gnome.org/gsf/ and source code at github.com/GNOME/libgsf. Or install the package named libgsf-bin in Debian/Ubuntu for access to the “gsf” binary (man page) that can inspect and dump contents from a .msg file. This is quite rudimentary if you just want to read the content.

MSGViewer (java)

MSGViewer is a stand alone java application that can display/convert different mail-format, including .msg. On GitHub I found a fork which claims to provide further features than the original.

msgconv (perl)

msgconv (github source) is a perl script that converts a .msg file into what it dubs as standard RFC 822 format (aka. RFC 2822/RFC 5322 format) in a file with extension “.eml” (email) format. This seems to be compatible (or identical?) with Thunderbird’s “.eml” format, it reads the converted files at least, as does Outlook itself. The fileformat.com entry for eml seems to support this.

Install as package “libemail-outlook-message-perl” on Debian/Ubuntu.

$ file test.msg
test.msg: Composite Document File V2 Document, Cannot read section info
$ msgconvert test.msg
$ file test.eml
test.eml: UTF-8 Unicode text, with very long lines, with CRLF line terminators

Conclusion

I prefer the approach of using a standard format for storing data, so I ended up accepting the .eml from msgconv which I propagated to other recipients of the file.

NOTE 2024-08-31: see also recent discussion I was involved in on a Mosquitto github issue about a similar need of generating credentials for use on systems where installed software is very constrained. As this pops up continually I’ll put on my tinkering TODO (which means no  promises or guarantees at all(!)) to look into and potentially contribute a suite of utilities to do this using different tools, currently I guess PHP, OpenSSL and maybe Python would be a good start. Feel free to also ping me if this would be something you are interested in.

NOTE 2021-07-11: prompted by an email from a reader the approach described below was refined into a script which can be used as a (sort of) drop-in replacement for mosquitto_passwd, find it on sourcehut in my hometools repository as mosquitto_passwd.php (first edition also on paste.sr.ht). Even though it generates a pre-2.0 plain hash using SHA512 this is still parsable by 2.0 and later. However, 2.0 adds support for PBKDF2 aka. RFC2898 on top of SHA512 with 20.000 iterations which is also the default generated by 2.0+ mosquitto_passwd, but the broker is still able to parse previous SHA512 hashes (notice the algorithm identifier after <username>:$, 6 for SHA512, 7 for PBKDF2-SHA512, similar to common /etc/shadow convention and extensions to implementations of POSIX crypt()).

Here is a delayed write-up of my involvement in a question posted to the liberally licensed MQTT broker (server) Mosquitto’s developer list list about how to generate authentication tokens programmatically. It kicked the curious cat in me which propelled a journey into the backyards of the C code for the mosquitto_passwd tool which normally is used for this purpose. This resulted in the proof of concept PHP implementation outlined in my answer on the list which is reproduced below.

MQTT (once Message Queue Telemetry Transport) is a lightweight publish/subscribe protocol intended for communication between low power, low bandwidth embedded devices. These days it is commonly hyped as a holy grail in the religion of IoT. The protocol was originally developed by IBM but is now a standard overseen by the OASIS standardization organization which also has the OpenDocument standard (ODF, think Open/Libre-Office) under its wings. According to Wikipedia MQTT is used behind the scenes of Facebook Messenger, OpenStack and Amazon’s IoT services.

For further practical use of the concept outlined you would need to produce a random 16 byte base64 encoded salt to feed into the hasher, that could be done using something like; $salt_base64=base64_encode(openssl_random_pseudo_bytes(12));

If your need a shrink wrapped solution to this you could try to ping me.

Hi Srinivas.

On 2016-07-26 12:49, Srinivas Pokala wrote:
> Username successfully created using linux command with: 
> "mosquitto_passwd /etc/mosquitto/passwd guest".
> I need to create same with php or javascript how?

Looking at the source of mosquitto_passwd
(https://github.com/eclipse/mosquitto/blob/master/src/mosquitto_passwd.c)
basically all it does to generate the resulting line you see in the
password file is:

1) draw a random 12 byte binary salt
2) hash the combination of password and salt using sha512
3) write username, base64 encoded salt, base64 encoded hash in one line

A PHP implementation would use something like this ($salt is fixed for
demonstration purposes, it ought to be random in production);

---
$username="Bitten";
$password="Insect";
$salt_base64="spicychilinstuff";
$salt=base64_decode($salt_base64);
$hash=hash("sha512",$password.$salt, true);
$hash_base64=base64_encode($hash);
echo($username.":$6$".$salt_base64."$".$hash_base64."\n");
---

Comparing against mosquitto_passwd using a one-liner (uses the base64
salt from output to be able to correlate the two);

---
$ mosquitto_passwd -b ~/mosq_passwd_test Bitten Insect
$ cat ~/mosq_passwd_test
Bitten:$6$mfJ0Eq3rIDLKG33r$gkiIlz80JA6Pq9OtGhasIsx7L2vf0APdZH77+thmNW2Zp5vE1d/dAi5TjbfO9mZpKHLh38Oem1ic072rSO328g==

$ php -r '$username="Bitten"; $password="Insect";
$salt_base64="mfJ0Eq3rIDLKG33r"; $salt=base64_decode($salt_base64);
$hash=hash("sha512",$password.$salt, true);
$hash_base64=base64_encode($hash);
echo($username.":$6$".$salt_base64."$".$hash_base64."\n");'
Bitten:$6$mfJ0Eq3rIDLKG33r$gkiIlz80JA6Pq9OtGhasIsx7L2vf0APdZH77+thmNW2Zp5vE1d/dAi5TjbfO9mZpKHLh38Oem1ic072rSO328g==
---

As can be seen, the PHP generated password line are identical to the
mosquitto_passwd generated.

I have also successfully tested authentication against the mosquitto
broker with PHP generated users. One caveat is that the above can
generate a salt of arbitrary length, but the broker must see a 12 byte
binary salt (16 byte base64) or authentication will fail.

Note however, that this hasn't been tested on more than a few
username/password pairs, there might be other issues lurking.

Regards,
-- 
Mikkel

Redigeret 2018-08-03: rettet døde links

Min kommentar til LoRa og LoRaWANs åbenhed i forbindelse med artikel i Version2 med overskriften “TDC holder fast i proprietær IoT-standard – andre kører billig open source”, som nok mest minder om et bidrag til “open washing” af LoRa/LoRaWAN-teknologien. Se mere om open washing i denne blogpost fra Open Knowledge Foundation Denmark (Engelsk udgave).

Det er noget misvisende at snakke om at LoRa eller LoRaWAN er “open source”. Man er i hvert fald nødt til at skelne hårdt mellem LoRa og LoRaWAN.

LoRa specificerer radiomodulationen i luften, og det er en proprietær teknologi udviklet af virksomheden Semtech, som både har varemærkebeskyttet og patenteret den (EPO-patent);

LoRa is a proprietary spread spectrum modulation scheme that is derivative of Chirp Spread Spectrum modulation  (CSS)

Kilde: Semtech AN1200.22, LoRa(TM) Modulation Basics (https://www.semtech.com/uploads/documents/an1200.22.pdf, afsnit 1)

LoRaWAN er en åben protokol der anvender LoRa som transmissionsmedie, men som yderligere definerer hvordan flere enheder der alle kan snakke LoRa kan fungere sammen i et netværk. Den specificeres af et samarbejde mellem mange interessenter i LoRa Alliance, herunder også Semtech.

Q: What is LoRaWAN(TM)?
A: The LoRa modulation is the pshysical layer (PHY), and LoRaWAN is a MAC protocol for a high capacity, long range star network that the LoRa Alliance is standardizing for Low Power WideArea Networks (LPWAN).

Kilde: Semtech LoRa® FAQ (http://www.semtech.com/wireless-rf/lora/LoRa-FAQs.pdf, spgm. 3)

En mere teknisk beskrivelse kan findes her; https://www.lora-alliance.org/What-Is-LoRa/Technology (from). LoRaWAN-specifikationen selv er næsten frit tilgængelig, det er kun folk med offentlige email-adresser der ikke må få den (dette er senere ændret, pr. 2018-08-03 er den frit tilgængelig).

For mig ser det dog ud som om Semtech og LoRa Alliance helt bevidst mudrer sondringen mellem LoRa og LoRaWAN, og synes det lugter af at det udelukkende er Semtech der kan og må producere silicium der implementerer LoRa. Selvom jeg ikke har kunnet finde steder det bliver sagt helt eksplicit.

Microchip og andre har produkter der snakker LoRa, men Semtech ser ud til altid at være med inde over, så jeg vil tro det stadig er en chip fra deres fabrik der ligger til grund for LoRa-funktionaliteten.

Der er dog folk der har kigget i sprækkerne på LoRa;

• https://revspace.nl/DecodingLora
• https://myriadrf.org/blog/lora-modem-limesdr/
• http://www.jailbreaksecuritysummit.com/s/Reversing-Lora-Knight.pdf

Spændende hvad der sker på dette felt. En mere fri og uafhængig infrastruktur i åbne licensbånd skal da være så velkommen.

Mikkel