WARNING: as OSM user “mmd” wisely points out (in comment to OSM diary for this post) the sort of thing described here is dangerous to do by hand, and should not be done on the main production instances (there are testing instances for playing around with the API, a little documentation here). He also points out that the feature packed JOSM editor actually supports continuing a changeset regardless of where it has been initiated. So if you just need to continue working on a changeset (but remember the 1 hour idle timeout) be sure to check out the JSOM upload documentation. Thanks mmd, for being the sane voice ;).
During an editing session the Android OpenStreetMap editor Vespucci crashed on me, which made the mapping UI unusable (objects greyed out and unable to select or edit, had to purge all data to recover functionality). Luckly, I could still navigate the menus, upload changes and opt to not close the changeset. Now, I had long wondered whether the OSM API allowed to continue amending changes to an open changeset from some other editor, so the quest began.
I had the intention of adding a tag representing the name of a building in Esbjerg known as “ISC Huset” (ISC is an engineering consultancy, see wikipedia, and more about the construction), which houses a number of healtcare clinics. The building’s address is Borgergade 70, 6700 Esbjerg (current address node).
This blog post willl attempt to actually add the tag by hand on the command line.
References to OSM objects used:
Option summary for GNU wget used to do HTTP requests below:
NOTE: I’ve broken some long output lines replicated below to make it fit the blog, but inserted an escape character (\) before the inserted newline to help copy’n’pasting.
Lets start by looking at the changeset’s metadata.
This can be done by issuing an unauthenticated GET request to the “/api/0.6/changeset/<changeset id>” endpoint.
Note that the ‘changeset’ element has the attribute ‘open=”true”‘ required to be able to modify the changeset. The editor used to create the changeset needs to have done this without explicitly closing it (Vespucci & JOSM closes by default but can be configured not to, iD always closes).
$ wget -nv -O- http://api.openstreetmap.org/api/0.6/changeset/88490797 <?xml version="1.0" encoding="UTF-8"?> <osm version="0.6" generator="CGImap 0.8.3 (802 thorn-02.openstreetmap.org)" copyright="OpenStreetMap and contributors"\ attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/"> <changeset id="88490797" created_at="2020-07-25T09:51:30Z" open="true" user="mikini" uid="2051"\ min_lat="55.4654072" min_lon="8.4378053" max_lat="55.5258532" max_lon="8.4639026" comments_count="0" changes_count="68"> <tag k="source" v="survey;research"/> <tag k="locale" v="da-DK"/> <tag k="created_by" v="Vespucci 14.1.4.0"/> <tag k="comment" v="Details at Klevestien & Borgergade in Esbjerg and Tarp."/> </changeset> </osm> 2020-07-25 14:26:52 URL:http://api.openstreetmap.org/api/0.6/changeset/88490797 [709] -> "-" [1] $
To test that the changeset is open, and that we can authenticate correctly, lets try amending it with an empty osmChange structure.
This can be done by issuing an authenticated POST request to the “/api/0.6/changeset/<changeset id>/upload” endpoint.
This also seem to reset the 60 minute timer used for auto-closing the changeset (see mention of “idle timeout” in changeset wiki article).
$ wget -nv -O- --user=mikini --ask-password https://api.openstreetmap.org/api/0.6/changeset/88490797/upload --post-data="<osmChange></osmChange>" Password for user ‘mikini’: Authentication selected: Basic realm="Web Password", charset="UTF-8" <?xml version="1.0" encoding="UTF-8"?> <diffResult version="0.6" generator="CGImap 0.8.3 (8531 thorn-03.openstreetmap.org)" copyright="OpenStreetMap and contributors"\ attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/"/> 2020-07-25 14:27:20 URL:https://api.openstreetmap.org/api/0.6/changeset/88490797/upload [278] -> "-" [1] $
The API’s “changeset/<changeset id>/upload” method supports only modifications encoded in the osmChange format which requires changes to be described as complete way/node/relation objects. That is, you can not ask the API to “add this tag to this way”, you need instead to describe the modified way completely saying “this way now looks like this”, including anything (like node references or existing tags) that was not modified. So to make a modificatino to the building’s way we need to retrieve the current version, modify it as needed and upload the complete new way.
Thus the procedure contains these steps;
This can be done by issuing an unauthenticated GET request to the “/api/0.6/way/<way id>” endpoint.
The stdin splitter ‘tee’ is used here to both show the result in terminal and put it into file 185369466_v3.osc that we can use for amending the way with the wanted modifications.
$ wget -nv -O- http://api.openstreetmap.org/api/0.6/way/185369466|tee 185369466_v3.osc <?xml version="1.0" encoding="UTF-8"?> <osm version="0.6" generator="CGImap 0.8.3 (28697 thorn-01.openstreetmap.org)" copyright="OpenStreetMap and contributors"\ attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/"> <way id="185369466" visible="true" version="3" changeset="84400254" timestamp="2020-04-30T09:35:21Z" user="mikini" uid="2051"> <nd ref="1959614623"/> <nd ref="1959614727"/> <nd ref="6299449794"/> <nd ref="1959614650"/> <nd ref="1959614630"/> <nd ref="6299449793"/> <nd ref="1959614765"/> <nd ref="7466482063"/> <nd ref="7466482064"/> <nd ref="7466482065"/> <nd ref="7466482062"/> <nd ref="1959614729"/> <nd ref="1959614623"/> <tag k="building" v="yes"/> </way> </osm> 2020-07-25 14:54:30 URL:http://api.openstreetmap.org/api/0.6/way/185369466 [769] -> "-" [1] $
Now we need to build an osmChange file out of the existing <way>…</way> element from the output above describing the wanted building. This involves;
Use your favorite editor (emacs would be my preference) to load the 185369466_v3.osm file, make the modifications and save it as 185369466_v4.osc. OSM tags are a XML empty-element tags containing the OSM tag’s key and value in the “k” and “v” attributes, thus the “name” tag of the building I needed to add would be ‘<tag k=”name” v=”ISC Huset”/>’, I also added some other related tags (“source:name” and “website”).
The finished .osc file now looks like this;
$ cat 185369466_v4.osc <osmChange> <modify> <way id="185369466" visible="true" version="3" changeset="88490797" timestamp="2020-04-30T09:35:21Z" user="mikini" uid="2051"> <nd ref="1959614623"/> <nd ref="1959614727"/> <nd ref="6299449794"/> <nd ref="1959614650"/> <nd ref="1959614630"/> <nd ref="6299449793"/> <nd ref="1959614765"/> <nd ref="7466482063"/> <nd ref="7466482064"/> <nd ref="7466482065"/> <nd ref="7466482062"/> <nd ref="1959614729"/> <nd ref="1959614623"/> <tag k="building" v="yes"/> <tag k="name" v="ISC Huset"/> <tag k="source:name" v="sign;website"/> <tag k="website" v="https://www.isc.dk/isc-huset-esbjerg/"/> </way> </modify> </osmChange> $
Wdiff’ing against the .osm source shows exactly what changed (additions between “{+” & “+}”, removals between “[-” & “-]”);
$ wdiff 185369466_v3.osm 185369466_v4.osc [-<?xml version="1.0" encoding="UTF-8"?> <osm version="0.6" generator="CGImap 0.8.3 (28697 thorn-01.openstreetmap.org)" copyright="OpenStreetMap and contributors"\ attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">-]{+<osmChange> <modify>+} <way id="185369466" visible="true" version="3" [-changeset="84400254"-] {+changeset="88490797"+} timestamp="2020-04-30T09:35:21Z" user="mikini" uid="2051"> <nd ref="1959614623"/> <nd ref="1959614727"/> <nd ref="6299449794"/> <nd ref="1959614650"/> <nd ref="1959614630"/> <nd ref="6299449793"/> <nd ref="1959614765"/> <nd ref="7466482063"/> <nd ref="7466482064"/> <nd ref="7466482065"/> <nd ref="7466482062"/> <nd ref="1959614729"/> <nd ref="1959614623"/> <tag k="building" v="yes"/> {+<tag k="name" v="ISC Huset"/> <tag k="source:name" v="sign;website"/> <tag k="website" v="https://www.isc.dk/isc-huset-esbjerg/"/>+} </way> [-</osm>-] {+</modify> </osmChange>+} $
Now, we’ll again use the changeset upload method but this time supplying our actual osmChange elemet in the .osc file.
The output is a bit elaborate, because I had enabled full output from wget while debugging what changes to the <way> element was needed for the server to accept the upload (only the “changeset” attribute needs to match the open changeset as outlined in the “Modify building data” above). I’ve highligted the actual server response telling that the changes were accepted and way #185369466 is now at v4.
$ wget -S -O- --user=mikini --ask-password https://api.openstreetmap.org/api/0.6/changeset/88490797/upload --post-file=185369466_v4.osc Password for user ‘mikini’: --2020-07-25 15:44:41-- https://api.openstreetmap.org/api/0.6/changeset/88490797/upload Resolving api.openstreetmap.org (api.openstreetmap.org)... 130.117.76.12, 130.117.76.13, 130.117.76.11, ... Connecting to api.openstreetmap.org (api.openstreetmap.org)|130.117.76.12|:443... connected. HTTP request sent, awaiting response... HTTP/1.1 401 Unauthorized Date: Sat, 25 Jul 2020 13:44:41 GMT Server: Apache/2.4.29 (Ubuntu) Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly" WWW-Authenticate: Basic realm="Web Password", charset="UTF-8" Cache-Control: no-cache Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly" Content-Length: 22 Content-Type: text/plain; charset=utf-8 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Authentication selected: Basic realm="Web Password", charset="UTF-8" Reusing existing connection to api.openstreetmap.org:443. HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Sat, 25 Jul 2020 13:44:42 GMT Server: Apache/2.4.29 (Ubuntu) Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly" Content-Encoding: identity Cache-Control: private, max-age=0, must-revalidate Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly" Vary: Accept-Encoding Content-Type: application/xml; charset=utf-8 Keep-Alive: timeout=5, max=99 Connection: Keep-Alive Transfer-Encoding: chunked Length: unspecified [application/xml] Saving to: ‘STDOUT’ - [<=> ] 0 --.-KB/s <?xml version="1.0" encoding="UTF-8"?> <diffResult version="0.6" generator="CGImap 0.8.3 (8537 thorn-03.openstreetmap.org)" copyright="OpenStreetMap and contributors"\ attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/"> <way old_id="185369466" new_id="185369466" new_version="4"/> </diffResult> - [ <=> ] 353 --.-KB/s in 0s 2020-07-25 15:44:42 (20,3 MB/s) - written to stdout [353] $
That was it, the building is now named in OSM, in a changset amended by hand.
Take a look at https://www.openstreetmap.org/way/185369466/.
Inspired by a John Sullivan (of FSF) tweet asking about opinions on the new and shiny MS(TM) GitHub(TM) CLI(TM) tool named “gh”(TM) I wanted to try it out on one of my Ubuntu LTS systems (16.04).
I’ve always disliked the proprietary and centralised monoculture of github, especially after that thing with MS, so I’ve mostly avoided using the service for code I produce myself. If interested in independence and decentralisation maybe you should read up on that long fabled subject? Thus I’ve never even tried the predecessor “hub”(TM) tool (main page), and exactly how it differs from this new “CLI” thing is not obviously apparent from the communications.
Nevertheless, the “gh” tool, officially dubbed GitHub CLI, being distributed under MIT license and written in go(lang) (afterwards, found out that this is indeed also applicable to the predecessing “hub” tool) made me curious enough to go see what they’re up to. Maybe its not that sinister a plot and they really just want to improve the independent and freely available mechanisms for source code storage, distribution and maintenance (I wish)?
A fairly recent (2019-09) version of go, >=1.13, is required so some gymnastics are needed to make stuff work on older distros like Ubuntu’s LTS (with 5 years support from Canonical). Anyway, this is what I did to build a useable “gh” ELF executable (20 MiB!, se below), this is from source, no precompiled stuff, no snaps. It was done on Ubuntu 16.04 (xenial) but I’ve been somewhat careful to inspect that things ought to be good for 18.04 (bionic) too. Non-essential output from the commands is largely discarded here. See Gory Details section below for the full monty.
$ sudo add-apt-repository ppa:longsleep/golang-backports $ sudo apt update $ sudo apt install golang $ go version go version go1.13.4 linux/amd64
$ cd $ git clone https://github.com/cli/cli .githubcli $ cd .githubcli $ make
$ echo 'export PATH="$HOME/.githubcli/bin:$PATH"' >> ~/.bash_profile $ bash --login $ gh Work seamlessly with GitHub from the command line. GitHub CLI is in early stages of development, and we'd love to hear your feedback at <https://forms.gle/umxd3h31c7aMQFKG7> Usage: gh [command] Available Commands: help Help about any command issue Create and view issues pr Create, view, and checkout pull requests repo Create, clone, fork, and view repositories Flags: --help Show help for command -R, --repo OWNER/REPO Select another repository using the OWNER/REPO format --version Show gh version Use "gh [command] --help" for more information about a command. subcommand is required $
Randomly picked one of my few github repos, it contains some js experiments on parsing NMEA formatted coordinates, rather uninterestingly empty.
user@host:~$ gh repo clone mikini/coordinates Notice: authentication required Press Enter to open github.com in your browser... <auth session in browser> Authentication complete. Press Enter to continue... Cloning into 'coordinates'... remote: Enumerating objects: 10, done. remote: Total 10 (delta 0), reused 0 (delta 0), pack-reused 10 Unpacking objects: 100% (10/10), done. Checking connectivity... done. user@host:~$
An ordinary OAuth request was spawned for authentication on https://github.com/login/oauth/authorize. The javascript executed in the browser seemed to interact with the gh tool through a local TCP connection on localhost:45454 which was the redirection target on authorisation (URL parameter: “redirect_uri=http://localhost:45454/callback”). The socket was rightfully not bound after the authentication had succeeded but this is definitely an attack vector, albeit hopefully only for locally running processes.
user@host:~$ cd coordinates/ user@host:~$:~/coordinates$ gh issue list Issues for mikini/coordinates There are no open issues miki@vcas-miki2:~/coordinates$ gh pr view no open pull requests found for branch "master" miki@vcas-miki2:~/coordinates$
miki@vcas-miki2:~/coordinates$ gh issue create Creating issue in mikini/coordinates ? Title Testing gh ? Body <Received> ? What's next? Preview in browser Opening github.com/mikini/coordinates/issues/new/ in your browser. miki@vcas-miki2:~/coordinates$ miki@vcas-miki2:~/coordinates$ gh issue create Creating issue in mikini/coordinates ? Title Testing gh tool ? Body <Received> ? What's next? [Use arrows to move, type to filter] > Preview in browser Submit Cancel https://github.com/mikini/coordinates/issues/1 miki@vcas-miki2:~/coordinates$
On the first issue creation interaction I selected “preview in browser” which opened a draft issue in the browser, and dumped me back into the shell on the command line. The second interaction actually resulted in #1.
The golang-backports PPA is maintained independently by Simon Eisenmann (credits: ~longsleep, @github, stdin.xyz) and carries packages up to:
There’s also an official PPA from the golang project (~gophers) which however only carries packages up to 1.11 for both.
user@host:~$ sudo add-apt-repository ppa:longsleep/golang-backports Golang 1.8, 1.9, 1.10, 1.11, 1.12, 1.13 and 1.14 PPA for Ubuntu More info: https://launchpad.net/~longsleep/+archive/ubuntu/golang-backports Press [ENTER] to continue or ctrl-c to cancel adding it gpg: keyring `/tmp/tmpvnuym97a/secring.gpg' created gpg: keyring `/tmp/tmpvnuym97a/pubring.gpg' created gpg: requesting key 56A3D45E from hkp server keyserver.ubuntu.com gpg: /tmp/tmpvnuym97a/trustdb.gpg: trustdb created gpg: key 56A3D45E: public key "Launchpad PPA for Simon Eisenmann" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK user@host:~$ user@host:~$ sudo apt update Hit:1 http://dk.archive.ubuntu.com/ubuntu xenial InRelease Get:2 http://dk.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB] ... ... Get:37 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [194 kB] Get:38 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 DEP-11 Metadata [2.468 B] Fetched 7.614 kB in 3s (2.346 kB/s) Reading package lists... Done Building dependency tree Reading state information... Done 419 packages can be upgraded. Run 'apt list --upgradable' to see them. user@host:~$ user@host:~$ apt list --upgradeable golang -a Listing... Done golang/xenial,xenial 2:1.13~1longsleep1+xenial all [upgradable from: 2:1.6-1ubuntu4] golang/xenial,xenial,now 2:1.6-1ubuntu4 all [installed,upgradable to: 2:1.13~1longsleep1+xenial] user@host:~$ user@host:~$ sudo apt install golang Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: golang-1.6 golang-1.6-doc golang-1.6-go golang-1.6-src Use 'sudo apt autoremove' to remove them. The following additional packages will be installed: golang-1.13 golang-1.13-doc golang-1.13-go golang-1.13-src golang-doc golang-go golang-src Recommended packages: golang-1.13-race-detector-runtime The following NEW packages will be installed golang-1.13 golang-1.13-doc golang-1.13-go golang-1.13-src The following packages will be upgraded: golang golang-doc golang-go golang-src 4 to upgrade, 4 to newly install, 0 to remove and 413 not to upgrade. Need to get 60,2 MB of archives. After this operation, 322 MB of additional disk space will be used. Do you want to continue? [Y/n] Get:1 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-1.13-src amd64 1.13.4-1longsleep1+xenial [12,7 MB] Get:2 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-1.13-go amd64 1.13.4-1longsleep1+xenial [44,9 MB] Get:3 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-1.13-doc all 1.13.4-1longsleep1+xenial [2.542 kB] Get:4 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-1.13 all 1.13.4-1longsleep1+xenial [25,2 kB] Get:5 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-src amd64 2:1.13~1longsleep1+xenial [3.838 B] Get:6 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-go amd64 2:1.13~1longsleep1+xenial [23,0 kB] Get:7 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang-doc all 2:1.13~1longsleep1+xenial [3.880 B] Get:8 http://ppa.launchpad.net/longsleep/golang-backports/ubuntu xenial/main amd64 golang all 2:1.13~1longsleep1+xenial [3.826 B] Fetched 60,2 MB in 5s (10,8 MB/s) Selecting previously unselected package golang-1.13-src. (Reading database ... 460737 files and directories currently installed.) Preparing to unpack .../golang-1.13-src_1.13.4-1longsleep1+xenial_amd64.deb ... Unpacking golang-1.13-src (1.13.4-1longsleep1+xenial) ... Selecting previously unselected package golang-1.13-go. Preparing to unpack .../golang-1.13-go_1.13.4-1longsleep1+xenial_amd64.deb ... Unpacking golang-1.13-go (1.13.4-1longsleep1+xenial) ... Selecting previously unselected package golang-1.13-doc. Preparing to unpack .../golang-1.13-doc_1.13.4-1longsleep1+xenial_all.deb ... Unpacking golang-1.13-doc (1.13.4-1longsleep1+xenial) ... Selecting previously unselected package golang-1.13. Preparing to unpack .../golang-1.13_1.13.4-1longsleep1+xenial_all.deb ... Unpacking golang-1.13 (1.13.4-1longsleep1+xenial) ... Preparing to unpack .../golang-src_2%3a1.13~1longsleep1+xenial_amd64.deb ... Unpacking golang-src (2:1.13~1longsleep1+xenial) over (2:1.6-1ubuntu4) ... Preparing to unpack .../golang-go_2%3a1.13~1longsleep1+xenial_amd64.deb ... Unpacking golang-go (2:1.13~1longsleep1+xenial) over (2:1.6-1ubuntu4) ... Preparing to unpack .../golang-doc_2%3a1.13~1longsleep1+xenial_all.deb ... Unpacking golang-doc (2:1.13~1longsleep1+xenial) over (2:1.6-1ubuntu4) ... Preparing to unpack .../golang_2%3a1.13~1longsleep1+xenial_all.deb ... Unpacking golang (2:1.13~1longsleep1+xenial) over (2:1.6-1ubuntu4) ... Processing triggers for man-db (2.7.5-1) ... Setting up golang-1.13-src (1.13.4-1longsleep1+xenial) ... Setting up golang-1.13-go (1.13.4-1longsleep1+xenial) ... Setting up golang-1.13-doc (1.13.4-1longsleep1+xenial) ... Setting up golang-1.13 (1.13.4-1longsleep1+xenial) ... Setting up golang-src (2:1.13~1longsleep1+xenial) ... Setting up golang-go (2:1.13~1longsleep1+xenial) ... Setting up golang-doc (2:1.13~1longsleep1+xenial) ... Setting up golang (2:1.13~1longsleep1+xenial) ... user@host:~$ user@host:~$ go version go version go1.13.4 linux/amd64 user@host:~$
user@host:~$ git clone https://github.com/cli/cli .githubcli Cloning into '.githubcli'... remote: Enumerating objects: 168, done. remote: Counting objects: 100% (168/168), done. remote: Compressing objects: 100% (100/100), done. remote: Total 5577 (delta 93), reused 122 (delta 67), pack-reused 5409 Receiving objects: 100% (5577/5577), 7.53 MiB | 2.61 MiB/s, done. Resolving deltas: 100% (3269/3269), done. Checking connectivity... done. user@host:~$ user@host:~$ cd .githubcli user@host:~/.githubcli (master=)$ make go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/mattn/go-colorable v0.1.6 go: downloading github.com/spf13/cobra v0.0.6 go: downloading gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71 go: downloading github.com/mattn/go-isatty v0.0.12 go: downloading golang.org/x/text v0.3.2 go: extracting github.com/mitchellh/go-homedir v1.1.0 go: extracting github.com/mattn/go-colorable v0.1.6 go: extracting github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: extracting github.com/spf13/cobra v0.0.6 go: extracting github.com/mattn/go-isatty v0.0.12 go: downloading golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae go: downloading golang.org/x/crypto v0.0.0-20200219234226-1ad67e1f0ef4 go: downloading github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b go: downloading github.com/briandowns/spinner v1.9.0 go: extracting gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71 go: downloading github.com/hashicorp/go-version v1.2.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/AlecAivazis/survey/v2 v2.0.7 go: downloading github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 go: downloading github.com/charmbracelet/glamour v0.1.1-0.20200304134224-7e5c90143acc go: extracting github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b go: extracting github.com/spf13/pflag v1.0.5 go: extracting github.com/hashicorp/go-version v1.2.0 go: downloading gopkg.in/yaml.v2 v2.2.8 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.0 go: downloading github.com/henvic/httpretty v0.0.4 go: extracting github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 go: extracting github.com/briandowns/spinner v1.9.0 go: extracting github.com/AlecAivazis/survey/v2 v2.0.7 go: downloading github.com/fatih/color v1.7.0 go: extracting github.com/charmbracelet/glamour v0.1.1-0.20200304134224-7e5c90143acc go: downloading github.com/yuin/goldmark v1.1.24 go: downloading github.com/microcosm-cc/bluemonday v1.0.2 go: extracting gopkg.in/yaml.v2 v2.2.8 go: extracting github.com/henvic/httpretty v0.0.4 go: downloading github.com/olekukonko/tablewriter v0.0.4 go: downloading github.com/muesli/reflow v0.1.0 go: extracting github.com/fatih/color v1.7.0 go: extracting github.com/microcosm-cc/bluemonday v1.0.2 go: extracting github.com/cpuguy83/go-md2man/v2 v2.0.0 go: extracting golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae go: extracting github.com/muesli/reflow v0.1.0 go: extracting github.com/olekukonko/tablewriter v0.0.4 go: downloading golang.org/x/net v0.0.0-20200219183655-46282727080f go: extracting github.com/yuin/goldmark v1.1.24 go: downloading github.com/alecthomas/chroma v0.7.2-0.20200304075647-34d9c7143bf5 go: downloading github.com/mattn/go-runewidth v0.0.8 go: downloading github.com/muesli/termenv v0.4.0 go: downloading github.com/russross/blackfriday/v2 v2.0.1 go: extracting golang.org/x/crypto v0.0.0-20200219234226-1ad67e1f0ef4 go: extracting github.com/mattn/go-runewidth v0.0.8 go: extracting github.com/russross/blackfriday/v2 v2.0.1 go: downloading github.com/shurcooL/sanitized_anchor_name v1.0.0 go: extracting github.com/alecthomas/chroma v0.7.2-0.20200304075647-34d9c7143bf5 go: extracting github.com/shurcooL/sanitized_anchor_name v1.0.0 go: extracting github.com/muesli/termenv v0.4.0 go: downloading github.com/dlclark/regexp2 v1.2.0 go: downloading github.com/lucasb-eyer/go-colorful v1.0.3 go: downloading github.com/google/goterm v0.0.0-20190703233501-fc88cf888a3f go: downloading github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 go: extracting golang.org/x/text v0.3.2 go: extracting github.com/google/goterm v0.0.0-20190703233501-fc88cf888a3f go: extracting github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 go: extracting golang.org/x/net v0.0.0-20200219183655-46282727080f go: extracting github.com/lucasb-eyer/go-colorful v1.0.3 go: extracting github.com/dlclark/regexp2 v1.2.0 go: finding github.com/spf13/cobra v0.0.6 go: finding github.com/AlecAivazis/survey/v2 v2.0.7 go: finding github.com/henvic/httpretty v0.0.4 go: finding github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: finding github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b go: finding gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71 go: finding github.com/spf13/pflag v1.0.5 go: finding github.com/hashicorp/go-version v1.2.0 go: finding github.com/mitchellh/go-homedir v1.1.0 go: finding github.com/briandowns/spinner v1.9.0 go: finding golang.org/x/text v0.3.2 go: finding github.com/cpuguy83/go-md2man/v2 v2.0.0 go: finding github.com/charmbracelet/glamour v0.1.1-0.20200304134224-7e5c90143acc go: finding github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 go: finding github.com/mattn/go-colorable v0.1.6 go: finding gopkg.in/yaml.v2 v2.2.8 go: finding github.com/russross/blackfriday/v2 v2.0.1 go: finding github.com/mattn/go-isatty v0.0.12 go: finding github.com/fatih/color v1.7.0 go: finding golang.org/x/crypto v0.0.0-20200219234226-1ad67e1f0ef4 go: finding github.com/muesli/termenv v0.4.0 go: finding golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae go: finding github.com/shurcooL/sanitized_anchor_name v1.0.0 go: finding github.com/yuin/goldmark v1.1.24 go: finding github.com/alecthomas/chroma v0.7.2-0.20200304075647-34d9c7143bf5 go: finding github.com/google/goterm v0.0.0-20190703233501-fc88cf888a3f go: finding github.com/dlclark/regexp2 v1.2.0 go: finding github.com/lucasb-eyer/go-colorful v1.0.3 go: finding github.com/muesli/reflow v0.1.0 go: finding github.com/microcosm-cc/bluemonday v1.0.2 go: finding github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 go: finding github.com/olekukonko/tablewriter v0.0.4 go: finding github.com/mattn/go-runewidth v0.0.8 go: finding golang.org/x/net v0.0.0-20200219183655-46282727080f user@host:~/.githubcli (master=)$
user@host:~/.githubcli (master=)$ ls -l bin/gh -rwxrwxr-x 1 user user 21097487 Mar 4 18:03 bin/gh user@host:~/.githubcli (master=)$ user@host:~/.githubcli (master=)$ file bin/gh bin/gh: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, not stripped user@host:~/.githubcli (master=)$ user@host:~/.githubcli (master=)$ ldd bin/gh linux-vdso.so.1 => (0x00007ffdc4329000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb1018c2000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb1014f8000) /lib64/ld-linux-x86-64.so.2 (0x00007fb101adf000) user@host:~/.githubcli (master=)$ user@host:~/.githubcli (master=)$ nm -D bin/gh U abort 00000000004fd140 T _cgo_panic 000000000045bf10 T _cgo_topofstack 00000000004fd190 T crosscall2 U __errno_location U __fprintf_chk U fputc U free U freeaddrinfo U fwrite U gai_strerror U getaddrinfo U getnameinfo U malloc U mmap U munmap U nanosleep U pthread_attr_destroy U pthread_attr_getstacksize U pthread_attr_init U pthread_cond_broadcast U pthread_cond_wait U pthread_create U pthread_detach U pthread_mutex_lock U pthread_mutex_unlock U pthread_sigmask U setenv U sigaction U sigaddset U sigemptyset U sigfillset U sigismember U __stack_chk_fail U stderr U strerror U unsetenv U __vfprintf_chk user@host:~/.githubcli (master=)$
user@host:~/.githubcli (master=)$ cd user@host:~$ echo 'export PATH="$HOME/.githubcli/bin:$PATH"' >> ~/.bash_profile user@host:~$ bash --login
user@host:~$ gh Work seamlessly with GitHub from the command line. GitHub CLI is in early stages of development, and we'd love to hear your feedback at <https://forms.gle/umxd3h31c7aMQFKG7> Usage: gh [command] Available Commands: help Help about any command issue Create and view issues pr Create, view, and checkout pull requests repo Create, clone, fork, and view repositories Flags: --help Show help for command -R, --repo OWNER/REPO Select another repository using the OWNER/REPO format --version Show gh version Use "gh [command] --help" for more information about a command. subcommand is required user@host:~$
Came by Esbjerg Harbour on September 24th 2019 and saw what was obviously a cable ship docked at the quay. A giant ship and I immediately thought that the mermaid might be closing in on Jutland. Some quick drive-by pictures and vessel details below:
New sighting on 2019-11-03:
JV article about ship and ongoing upgrades causing noise.
Will investigate later and amend the post (promise!).
$ free; find . -size +25M -exec bash -c 'ls -l {}; eog {}; echo' \;; free total used free shared buff/cache available Mem: 7746692 5767656 843228 317764 1135808 1271048 Swap: 7950332 5113012 2837320 -rw-rw-r-- 1 miki miki 39926496 Jun 10 19:25 ./20190609_184124.jpg ** (eog:23320): CRITICAL **: eog_reload_plugin_activate: assertion 'G_IS_MENU (model)' failed eog: ../../../../src/cairo-xlib-surface-shm.c:619: _cairo_xlib_shm_pool_create: Assertion `*ptr != ((void *)0)' failed. bash: line 1: 23320 Aborted (core dumped) eog ./20190609_184124.jpg -rw-rw-r-- 1 miki miki 38286099 Jun 10 19:26 ./20190609_183714.jpg ** (eog:23332): CRITICAL **: eog_reload_plugin_activate: assertion 'G_IS_MENU (model)' failed eog: ../../../../src/cairo-xlib-surface-shm.c:619: _cairo_xlib_shm_pool_create: Assertion `*ptr != ((void *)0)' failed. bash: line 1: 23332 Aborted (core dumped) eog ./20190609_183714.jpg -rw-rw-r-- 1 miki miki 36181801 Jun 9 17:13 ./20190609_160437.jpg ** (eog:23343): CRITICAL **: eog_reload_plugin_activate: assertion 'G_IS_MENU (model)' failed ** (eog:23343): CRITICAL **: eog_reload_plugin_deactivate: assertion 'G_IS_MENU (menu)' failed -rw-rw-r-- 1 miki miki 39059177 Jun 10 19:25 ./20190609_184146.jpg ** (eog:23354): CRITICAL **: eog_reload_plugin_activate: assertion 'G_IS_MENU (model)' failed eog: ../../../../src/cairo-xlib-surface-shm.c:619: _cairo_xlib_shm_pool_create: Assertion `*ptr != ((void *)0)' failed. bash: line 1: 23354 Aborted (core dumped) eog ./20190609_184146.jpg total used free shared buff/cache available Mem: 7746692 5773448 495604 659944 1477640 922956 Swap: 7950332 5112700 2837632 $
Lidt flere dråber i Spørgsmål&Svar-kategorien fra Facebook.
Når jeg endelig forvilder mig derind, og går på opdagelse i de tekniske grupper, ender det ofte med at får jeg skrevet en fristil i forsøget på at hjælpe.
Denne gang en snak om Android på “PC”, og basale digitale processeringsbehov for den almindelige dansker.
Stillet i gruppen “Danske Android Brugere “:
Er der nogen som ved, om man kan få en pc med Android system?
Mit svar:
Som andre omtaler, kan der fås en uofficiel variant af det frie styresystem Android til x86-arkitekturen (den gængse Intel/AMD-baserede computer kendt som “personlig computer”). Det projekt lever på https://www.android-x86.org/. Installationsvejledning på engelsk er på https://www.android-x86.org/installhowto.html. Man kan både installere som multiboot på samme disk som et eksisterende operativsystem, starte fra en ekstern disk (USB-medie, cd/dvdrom e.l.) eller evt. køre i en virtuel maskine på et eksisterende operativsystem (VirtualBox,QEMU/KVM, VMware Player/Workstation). Jeg har kun erfaring med livedisk boot fra USB og VM, og der synes jeg ikke altid tingene spiller perfekt, så forvent ikke en helt problemfri oplevelse. “Android på PC” er på kanten af noget understøttet, hvor man ofte er på egen hånd. Nogle af folkene bag Andoid-x86 forsøgte at lave en kommerciel forretning på det, hvor det var tanken at sælge det som færdige hardwareenheder, RemixOS – https://en.wikipedia.org/wiki/Remix_OS, men det gik ikke så godt og er lukket ned igen. Hvis behovet bare er “en bærbar computer uden for meget vrøvl”, så er en færdig Chromebook med Chrome OS (der ligesom Android også er bygget på GNU/Linux) eller noget af det der dyre Frugt-udstyr nok det mest tilgængelige (men jeg fornemmer at pris også kunne være en faktor?). Til forskel fra traditionelle operativsystemer til computere, er Chrome OS dog møntet specifikt på at få dig til at bruge Googles webbaserede tjenester (mere om softwaren bag på https://www.chromium.org/chromium-os), så hverken software eller hardware er beregnet til at lagre data på selve enheden, og der er sjældent meget diskplads tilgængelig til f.eks. billeder. Et hurtigt kig på markedet identificerer Acer Chromebook 15 (https://www.edbpriser.dk/produkt/acer-chromebook-15-cb515-1h-c7kg/) og Lenovo S330 (https://www.edbpriser.dk/produkt/lenovo-chromebook-s330-81jw-3292492/) som populære Chromebook i 2k kr.-klassen, men kender ikke maskinerne specifikt. God jagt :).
Som andre omtaler, kan der fås en uofficiel variant af det frie styresystem Android til x86-arkitekturen (den gængse Intel/AMD-baserede computer kendt som “personlig computer”).
Det projekt lever på https://www.android-x86.org/. Installationsvejledning på engelsk er på https://www.android-x86.org/installhowto.html. Man kan både installere som multiboot på samme disk som et eksisterende operativsystem, starte fra en ekstern disk (USB-medie, cd/dvdrom e.l.) eller evt. køre i en virtuel maskine på et eksisterende operativsystem (VirtualBox,QEMU/KVM, VMware Player/Workstation).
Jeg har kun erfaring med livedisk boot fra USB og VM, og der synes jeg ikke altid tingene spiller perfekt, så forvent ikke en helt problemfri oplevelse.
“Android på PC” er på kanten af noget understøttet, hvor man ofte er på egen hånd. Nogle af folkene bag Andoid-x86 forsøgte at lave en kommerciel forretning på det, hvor det var tanken at sælge det som færdige hardwareenheder, RemixOS – https://en.wikipedia.org/wiki/Remix_OS, men det gik ikke så godt og er lukket ned igen.
Hvis behovet bare er “en bærbar computer uden for meget vrøvl”, så er en færdig Chromebook med Chrome OS (der ligesom Android også er bygget på GNU/Linux) eller noget af det der dyre Frugt-udstyr nok det mest tilgængelige (men jeg fornemmer at pris også kunne være en faktor?). Til forskel fra traditionelle operativsystemer til computere, er Chrome OS dog møntet specifikt på at få dig til at bruge Googles webbaserede tjenester (mere om softwaren bag på https://www.chromium.org/chromium-os), så hverken software eller hardware er beregnet til at lagre data på selve enheden, og der er sjældent meget diskplads tilgængelig til f.eks. billeder.
Et hurtigt kig på markedet identificerer Acer Chromebook 15 (https://www.edbpriser.dk/produkt/acer-chromebook-15-cb515-1h-c7kg/) og Lenovo S330 (https://www.edbpriser.dk/produkt/lenovo-chromebook-s330-81jw-3292492/) som populære Chromebook i 2k kr.-klassen, men kender ikke maskinerne specifikt.
God jagt :).
Needed some space on the disk.
Had some music, some of it were wav files.
Threw together this script to transmogify it into ogg using ffmpeg.
Statistics say that 24 wav files occupying 1,7 GiB was made into 24 ogg files occupying 105 MiB, a reduction to ~6% (du -h calculates 2^10 numbers but uses SI 10^3 prefixes..).
miki@filly:~/Music$ find . -iname "*.wav" -print0 | sed s/.wav/.ogg/g |du --files0-from=- -h -c|tail -1 105M total miki@filly:~/Music$ find . -iname "*.wav" -print0 | sed s/.wav/.wav/g |du --files0-from=- -h -c|tail -1 1,7G total miki@filly:~/Music$ find . -iname "*.wav" -print0 | sed s/.wav/.ogg/g |du --files0-from=- -h |wc -l 24 miki@filly:~/Music$
Install prerequisite ffmpeg and ffprobe utilities on apt distributions by doing “apt install ffmpeg”.
BEWARE: script deletes any wav files in current directory and below without warning if conversion to ogg is fine according to ffmpeg return code and stdout/err.
Manual confirmation of all deletions can be enabled by uncommenting line 8 (CONFIRM=..).
It could (very) easily be amended for conversion between any container or compression format supported by ffmpeg by replacing “wav” and “ogg” in glob pattern and OGG variable definition (for completeness also in echo output…).
miki@filly:~/wavtest$ cat ~/hometools/wav2ogg.sh #! /bin/bash # # Make .ogg out of all .wav in cwd and below, remove .wav! # # When converting remove only if conversion is ok. # ffprobe any present .ogg and check if playtime is same as wav, remove only if all is good. #CONFIRM=-i # confirm deletion? REQS="ffmpeg ffprobe" which $REQS >/dev/null if [ ! $? -eq 0 ]; then echo We need $REQS to do stuff! exit 1 fi shopt -s globstar nullglob nocaseglob # enable 2-star magic for f in **/*.wav; do OGG=$(dirname "$f")/$(basename "$f" wav)ogg echo -n "$f: " if [ ! -f "$OGG" ]; then echo -en "\e[33mno ogg\e[0m" CONVERT=1 else echo -en "\e[32malready there\e[0m" if [ ! "$(ffprobe "$f" 2>&1|grep Duration|cut -d, -f1)" = "$(ffprobe "$OGG" 2>&1|grep Duration|cut -d, -f1)" ]; then echo -en " - \e[31mogg corrupt\e[0m" CONVERT=1 rm $CONFIRM "$OGG" else OGGOK=1 fi fi if [ $CONVERT ]; then echo -en " - \e[33mconverting\e[0m" OUT=$(ffmpeg -loglevel error -i "$f" "$OGG" 2>&1) if [ ! $? -eq 0 ]; then echo -e " \e[31mfatal error!\e[0m\nffmpeg output was:\n\n$OUT\n" rm $CONFIRM "$OGG" elif [ ! -z "$OUT" ]; then echo -e " \e[31mnon-fatal error!\e[0m\nffmpeg output was:\n\n$OUT\n" else OGGOK=1 fi fi if [ $OGGOK ]; then echo -e " - all good removing wav" rm $CONFIRM "$f" fi done miki@filly:~/wavtest$
miki@filly:~/wavtest$ ls -lR .: total 59964 drwxrwxr-x 3 miki miki 4096 Jun 6 22:33 one -rw-rw-r-- 1 miki miki 61390864 Jun 6 22:32 Sordid Affair (Man Without Country Version).wav ./one: total 30720 drwxrwxr-x 3 miki miki 4096 Jun 6 22:34 two -rw-rw-r-- 1 miki miki 31451304 Jun 6 22:30 Vision Man MASTER.wav ./one/two: total 55352 -rw-rw-r-- 1 miki miki 56675660 Jun 6 22:34 Bye Barat - Going For Broke (UNKWON´s Autumn Remix) LIM_02.wav drwxrwxr-x 2 miki miki 4096 Jun 6 22:34 three ./one/two/three: total 375112 -rw-rw-r-- 1 miki miki 384108056 Jun 6 22:30 Rigoremix - Pornographe - Final2.wav miki@filly:~/wavtest$ du -hs . 509M . miki@filly:~/wavtest$
In real life, also in technicolor!
miki@filly:~/wavtest$ ~/hometools/wav2ogg.sh one/two/Bye Barat - Going For Broke (UNKWON´s Autumn Remix) LIM_02.wav: no ogg - converting - all good removing wav one/two/three/Rigoremix - Pornographe - Final2.wav: no ogg - converting - all good removing wav one/Vision Man MASTER.wav: no ogg - converting - all good removing wav Sordid Affair (Man Without Country Version).wav: no ogg - converting - all good removing wav miki@filly:~/wavtest$
miki@filly:~/wavtest$ ls -lR .: total 4952 drwxrwxr-x 3 miki miki 4096 Jun 6 22:35 one -rw-rw-r-- 1 miki miki 5066460 Jun 6 22:35 Sordid Affair (Man Without Country Version).ogg ./one: total 2528 drwxrwxr-x 3 miki miki 4096 Jun 6 22:34 two -rw-rw-r-- 1 miki miki 2580752 Jun 6 22:35 Vision Man MASTER.ogg ./one/two: total 4680 -rw-rw-r-- 1 miki miki 4785594 Jun 6 22:34 Bye Barat - Going For Broke (UNKWON´s Autumn Remix) LIM_02.ogg drwxrwxr-x 2 miki miki 4096 Jun 6 22:35 three ./one/two/three: total 19456 -rw-rw-r-- 1 miki miki 19922936 Jun 6 22:35 Rigoremix - Pornographe - Final2.ogg miki@filly:~/wavtest$ du -hs . 31M . miki@filly:~/wavtest$
ffmpeg does some nice metadata conversions out of the box;
miki@filly:~/Music$ ffprobe ~/"wavbak/Delayscape A Short While Rigolin Edit Audaccity.wav" 2>&1| grep Input -A 1000 Input #0, wav, from '/home/miki/wavbak/Delayscape A Short While Rigolin Edit Audaccity.wav': Metadata: title : Delayscape - A Short While (Rigolin Edit) artist : rigolin comment : Love Melody date : 2018 Duration: 00:06:33.97, bitrate: 3072 kb/s Stream #0:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, 2 channels, flt, 3072 kb/s miki@filly:~/Music$ ffprobe "./Delayscape - Heinz Beauvaix/Soundcloud/Delayscape A Short While Rigolin Edit Audaccity.ogg" 2>&1| grep Input -A 1000 Input #0, ogg, from './Delayscape - Heinz Beauvaix/Soundcloud/Delayscape A Short While Rigolin Edit Audaccity.ogg': Duration: 00:06:33.97, start: 0.000000, bitrate: 99 kb/s Stream #0:0: Audio: vorbis, 48000 Hz, stereo, fltp, 112 kb/s Metadata: ENCODER : Lavc56.60.100 libvorbis TITLE : Delayscape - A Short While (Rigolin Edit) ARTIST : rigolin comment : Love Melody DATE : 2018 miki@filly:~/Music$
Here’s a preliminary HOWTO from my recent external flashing of the BIOS ROM on a Lenovo X200 thinkpad (Wikipedia). The particular firmware flashed was a Libreboot build for this machine (instructions for X200 and details of the external flashing procedure) but anything goes (but anything may not be useful, though).
I’ll amend this HOWTO with more detailed instructions and pictures in the following days (warning: a prediction) to hopefully make it more complete and useful for the vary inhabitants of LibreBootLand.
A programmer kit was bought on AliExpress for $4.20 containing a USB programmer board and an SOIC-8 clip which ended up not being used as the particular X200 had a SOIC-16 chip so a separately ordered SOIC-16 chip ($3.11) was used.
Buy: https://www.aliexpress.com/item/32898599200.html @ $4.20
The programmer is based on the WCH CH341a chip which is an USB <-> seriel/parallel/uart interface. The manufacturer WCH being WinChipHead aka. WCH (Nanjing QinHeng Electronics Co.,Ltd) (maybe also aka. WCH-IC (Jiangsu Qinheng Co., Ltd)). There are lots of options for buying board varieties based on the CH341a chip, to get you started here is a BangGood search and an AliExpress search.
Boards like this has also been described by others including a deduced schematic, EEVblog critique of the I/O pin power on similar boards (not yet confirmed whether that is true for this programmer too, I guess so, but at least one flashing done without damage) and a mention on hackaday of other board types.
There are a bunch of downloads from the WCH site regarding the chip including a Chinese datasheet, no English language documentation seems to be available from the manufacturer however. There are some English editions of the datasheet to be found, of unknown origin. They seem plausible enough to use, though. Somebody has attempted to collect documentation about the chip in a Git repository.
Buy: https://www.aliexpress.com/item/32869145935.html @ $3.11
To attach physically to the Macronix MX25L6405D flash memory chip in a SOIC-16 package present on the X200 in question (words are that this is the norm although the board can be populated with a SOIC-8 too) a clip that matches the pins of the SOIC-16 package is needed. I bought the one mentioned above for $3.11 at random from AliExpress and this worked fine. In the pictures the wiring is hooked up correctly to the programmer to allow for flashing as described below.
6405 <-> CH341a
MISO<->MIOS (label error, should be MISO)
MOSI<->MOSI
CLK <-> CLK
CS <-> SS
GND <-> GND
First tried driving the the flash chip from VDD on ch341a but this was unsuccessful, no chip could be found, so the 6405 was hooked up to external 3.3v power supply with supply GND connected to GND on CH341a to align the ground potential between ch341a I/O supply and 6405 supply (important!).
To get the latest ECP (Embedded Controller Program) from Lenovo (no free alternative exists) containing software for the MCU controlling low level hardware like battery charging/keyboard/backlight stuff you need to update the BIOS which also updates the ECP. Most recent version for X200 is “BIOS: 3.22 / ECP: 1.07“. This is not needed if you already have these versions on the machine, check current versions by pressing ThinkVantage during boot and choosing “Enter Setup”. If your system has a Windows installation download and run the “BIOS Update Utility“executeable. Else you’ll need to get the “BIOS Update Bootable CD” and somehow get it on a CD and find a CD-ROM drive. Alternatively on a Linux system the CD file system can be extracted and added to Grub to be directly bootable. Below was done on an Ubuntu 16.04 system: $ sudo apt install genisoimage syslinux $ wget -q https://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/6duj48us.iso $ geteltorito 6duj48us.iso > 6duj48us.img Booting catalog starts at sector: 20 Manufacturer of CD: NERO BURNING ROM Image architecture: x86 Boot media type is: harddisk El Torito image starts at sector 27 and has 75776 sector(s) of 512 Bytes Image has been written to stdout …. $ sudo cp 6duj48us.im /boot $ sudo cp /usr/lib/syslinux/memdisk /boot $ sudo nano /etc/grub.d/40_custom <add lines below to the end of file, preserve the “exec tail…” line> menuentry “BIOS Update” { linux16 /memdisk initrd16 /6duj48us.im } $ sudo update-grub Reboot, press <left shift> key while booting to access Grub, choose BIOS Update menu entry and follow the Lenovo update procedure. To start flashing it requires both a connected power supply and also a working, non-exhausted battery (!) mounted in the machine. This is tiresome for owners of worn out batteries… Some notes about the flashing process can be found in the documentation of a patch set for the Lenovo BIOS.
To get the latest ECP (Embedded Controller Program) from Lenovo (no free alternative exists) containing software for the MCU controlling low level hardware like battery charging/keyboard/backlight stuff you need to update the BIOS which also updates the ECP. Most recent version for X200 is “BIOS: 3.22 / ECP: 1.07“. This is not needed if you already have these versions on the machine, check current versions by pressing ThinkVantage during boot and choosing “Enter Setup”.
If your system has a Windows installation download and run the “BIOS Update Utility“executeable. Else you’ll need to get the “BIOS Update Bootable CD” and somehow get it on a CD and find a CD-ROM drive. Alternatively on a Linux system the CD file system can be extracted and added to Grub to be directly bootable. Below was done on an Ubuntu 16.04 system:
$ sudo apt install genisoimage syslinux $ wget -q https://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/6duj48us.iso $ geteltorito 6duj48us.iso > 6duj48us.img Booting catalog starts at sector: 20 Manufacturer of CD: NERO BURNING ROM Image architecture: x86 Boot media type is: harddisk El Torito image starts at sector 27 and has 75776 sector(s) of 512 Bytes Image has been written to stdout …. $ sudo cp 6duj48us.im /boot $ sudo cp /usr/lib/syslinux/memdisk /boot $ sudo nano /etc/grub.d/40_custom <add lines below to the end of file, preserve the “exec tail…” line> menuentry “BIOS Update” { linux16 /memdisk initrd16 /6duj48us.im } $ sudo update-grub Reboot, press <left shift> key while booting to access Grub, choose BIOS Update menu entry and follow the Lenovo update procedure. To start flashing it requires both a connected power supply and also a working, non-exhausted battery (!) mounted in the machine. This is tiresome for owners of worn out batteries… Some notes about the flashing process can be found in the documentation of a patch set for the Lenovo BIOS.
$ sudo apt install genisoimage syslinux $ wget -q https://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/6duj48us.iso $ geteltorito 6duj48us.iso > 6duj48us.img Booting catalog starts at sector: 20 Manufacturer of CD: NERO BURNING ROM Image architecture: x86 Boot media type is: harddisk El Torito image starts at sector 27 and has 75776 sector(s) of 512 Bytes Image has been written to stdout …. $ sudo cp 6duj48us.im /boot $ sudo cp /usr/lib/syslinux/memdisk /boot $ sudo nano /etc/grub.d/40_custom <add lines below to the end of file, preserve the “exec tail…” line> menuentry “BIOS Update” { linux16 /memdisk initrd16 /6duj48us.im } $ sudo update-grub
Reboot, press <left shift> key while booting to access Grub, choose BIOS Update menu entry and follow the Lenovo update procedure. To start flashing it requires both a connected power supply and also a working, non-exhausted battery (!) mounted in the machine. This is tiresome for owners of worn out batteries…
Some notes about the flashing process can be found in the documentation of a patch set for the Lenovo BIOS.
sudo apt install flashrom
ch341a support in flashrom
Download the stable LibreBoot firmware: https://www.mirrorservice.org/sites/libreboot.org/release/stable/20160907/rom/grub/libreboot_r20160907_grub_x200_8mb.tar.xz
The brave will of course want to compile it themselves.
$ cd $ wget https://www.mirrorservice.org/sites/libreboot.org/release/stable/20160907/rom/grub/libreboot_r20160907_grub_x200_8mb.tar.xz --2019-06-07 07:35:21-- https://www.mirrorservice.org/sites/libreboot.org/release/stable/20160907/rom/grub/libreboot_r20160907_grub_x200_8mb.tar.xz Resolving www.mirrorservice.org (www.mirrorservice.org)... 212.219.56.184, 2001:630:341:12::184 Connecting to www.mirrorservice.org (www.mirrorservice.org)|212.219.56.184|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1632800 (1,6M) [application/x-xz] Saving to: ‘libreboot_r20160907_grub_x200_8mb.tar.xz’ libreboot_r20160907_grub_x200_8mb.tar.xz 100%[================================================================================>] 1,56M --.-KB/s in 0,1s 2019-06-07 07:35:21 (13,9 MB/s) - ‘libreboot_r20160907_grub_x200_8mb.tar.xz’ saved [1632800/1632800] $ tar tf libreboot_r20160907_grub_x200_8mb.tar.xz libreboot_r20160907_grub_x200_8mb/ libreboot_r20160907_grub_x200_8mb/x200_8mb_deqwertz_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_esqwerty_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_frazerty_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_frdvbepo_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_itqwerty_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_svenska_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_ukdvorak_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_ukqwerty_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_usdvorak_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_usqwerty_txtmode.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_deqwertz_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_esqwerty_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_frazerty_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_frdvbepo_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_itqwerty_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_svenska_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_ukdvorak_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_ukqwerty_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_usdvorak_vesafb.rom libreboot_r20160907_grub_x200_8mb/x200_8mb_usqwerty_vesafb.rom libreboot_r20160907_grub_x200_8mb/ChangeLog libreboot_r20160907_grub_x200_8mb/NEWS libreboot_r20160907_grub_x200_8mb/version libreboot_r20160907_grub_x200_8mb/versiondate $
As the MAC address of the ethnernet PHY is stored in the flash, yo have your X200 ethernet MAC address correspond to the sticker on the back of the machine, and also avoid a potential but improbable DHCP/ARP conflict, the MAC address from the label/ifconfig from the existing system must be embedded into the flash file that we are going to program into the flash chip.
For this a tool called ich9gen is needed, this is a part of the libreboot repository and we need to build it ourselves.
$ git clone https://notabug.org/libreboot/libreboot Cloning into 'libreboot'... remote: Counting objects: 29080, done. remote: Compressing objects: 100% (9855/9855), done. remote: Total 29080 (delta 18748), reused 27899 (delta 18057) Receiving objects: 100% (29080/29080), 63.90 MiB | 11.13 MiB/s, done. Resolving deltas: 100% (18748/18748), done. Checking connectivity... done. $ cd libreboot/projects/ich9gen/sources $ make gcc -I. -Wall -Wextra -g -std=c99 -c src/ich9deblob.c -o obj/ich9deblob.o gcc -I. -Wall -Wextra -g -std=c99 -c src/common/descriptor_gbe.c -o obj/common/descriptor_gbe.o gcc -I. -Wall -Wextra -g -std=c99 -c src/descriptor/descriptor.c -o obj/descriptor/descriptor.o gcc -I. -Wall -Wextra -g -std=c99 -c src/gbe/gbe.c -o obj/gbe/gbe.o gcc -I. -Wall -Wextra -g -std=c99 -c src/common/x86compatibility.c -o obj/common/x86compatibility.o gcc -I. -Wall -Wextra -g -std=c99 obj/ich9deblob.o obj/common/descriptor_gbe.o \ obj/common/x86compatibility.o obj/descriptor/descriptor.o obj/gbe/gbe.o \ -o ich9deblob gcc -I. -Wall -Wextra -g -std=c99 -c src/ich9gen.c -o obj/ich9gen.o gcc -I. -Wall -Wextra -g -std=c99 -c src/ich9gen/mkdescriptor.c -o obj/ich9gen/mkdescriptor.o gcc -I. -Wall -Wextra -g -std=c99 -c src/ich9gen/mkgbe.c -o obj/ich9gen/mkgbe.o gcc -I. -Wall -Wextra -g -std=c99 obj/ich9gen.o obj/ich9gen/mkdescriptor.o obj/ich9gen/mkgbe.o \ obj/common/descriptor_gbe.o \ obj/common/x86compatibility.o obj/descriptor/descriptor.o obj/gbe/gbe.o \ -o ich9gen gcc -I. -Wall -Wextra -g -std=c99 -c src/demefactory.c -o obj/demefactory.o gcc -I. -Wall -Wextra -g -std=c99 obj/demefactory.o obj/common/descriptor_gbe.o \ obj/common/x86compatibility.o obj/descriptor/descriptor.o obj/gbe/gbe.o \ -o demefactory $
Running ich9gen itself generates the flash descriptor (fd) header including possible configuration section where the MAC address is stored for the gigabit ethernet (gbe) PHY onboard the ICH9 chipset. When run six 12 KiB files for respectively 4, 8 and 16 MiB binary images and chipsets including (gbe) and excluding (nogbe) gigabit ethernet PHY are generated.
“aa:bb:cc:dd:ee:ff” in the commandline should be replaced with the actual 12 hex digits from the label on the machine or by running ifconfig on the machine using the existing Lenovo BIOS.
$ cd ~/libreboot_r20160907_grub_x200_8mb/ $ ~/libreboot/projects/ich9gen/sources/ich9gen --macaddress aa:bb:cc:dd:ee:ff You selected to change the MAC address in the Gbe section. This has been done. The modified gbe region has also been dumped as src files: mkgbe.c, mkgbe.h To use these in ich9gen, place them in src/ich9gen/ and re-build ich9gen. descriptor and gbe successfully written to the file: ich9fdgbe_4m.bin Now do: dd if=ich9fdgbe_4m.bin of=libreboot.rom bs=1 count=12k conv=notrunc (in other words, add the modified descriptor+gbe to your ROM image) descriptor and gbe successfully written to the file: ich9fdgbe_8m.bin Now do: dd if=ich9fdgbe_8m.bin of=libreboot.rom bs=1 count=12k conv=notrunc (in other words, add the modified descriptor+gbe to your ROM image) descriptor and gbe successfully written to the file: ich9fdgbe_16m.bin Now do: dd if=ich9fdgbe_16m.bin of=libreboot.rom bs=1 count=12k conv=notrunc (in other words, add the modified descriptor+gbe to your ROM image) descriptor successfully written to the file: ich9fdnogbe_4m.bin Now do: dd if=ich9fdnogbe_4m.bin of=yourrom.rom bs=1 count=4k conv=notrunc (in other words, add the modified descriptor to your ROM image) descriptor successfully written to the file: ich9fdnogbe_8m.bin Now do: dd if=ich9fdnogbe_8m.bin of=yourrom.rom bs=1 count=4k conv=notrunc (in other words, add the modified descriptor to your ROM image) descriptor successfully written to the file: ich9fdnogbe_16m.bin Now do: dd if=ich9fdnogbe_16m.bin of=yourrom.rom bs=1 count=4k conv=notrunc (in other words, add the modified descriptor to your ROM image)
$ cd ~/libreboot_r20160907_grub_x200_8mb/ $ cp -v x200_8mb_usqwerty_vesafb{,_customised}.rom 'x200_8mb_usqwerty_vesafb.rom' -> 'x200_8mb_usqwerty_vesafb_customised.rom' $ dd if=ich9fdgbe_8m.bin of=x200_8mb_usqwerty_vesafb_customised.rom bs=1 count=12k conv=notrunc 12288+0 records in 12288+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0,0299453 s, 410 kB/s $
flashrom -p ch341a_spi -c MX25L6405D -r rom1.bin flashrom -p ch341a_spi -c MX25L6405D -r rom2.bin flashrom -p ch341a_spi -c MX25L6405D -r rom3.bin flashrom -p ch341a_spi -c MX25L6405D -r rom4.bin flashrom -p ch341a_spi -c MX25L6405D -r rom5.bin cmp rom{1,2}.bin cmp rom{1,3}.bin cmp rom{1,4}.bin cmp rom{1,5}.bin
flashrom -p ch341a_spi -c MX25L6405D -r rom1.bin
flashrom -p ch341a_spi -c MX25L6405D -r rom2.bin
flashrom -p ch341a_spi -c MX25L6405D -r rom3.bin
flashrom -p ch341a_spi -c MX25L6405D -r rom4.bin
flashrom -p ch341a_spi -c MX25L6405D -r rom5.bin
cmp rom{1,2}.bin
cmp rom{1,3}.bin
cmp rom{1,4}.bin
cmp rom{1,5}.bin
$ sudo flashrom -p ch341a_spi -c MX25L6405D -w ~/libreboot_r20160907_grub_x200_8mb/x200_8mb_usqwerty_vesafb_customised.rom $
Tænk hvor meget trafik, processerings- og ventetid der ville være sparet hvis nogen gad at overveje hvad billeder på en hjemmeside anvendes til og tilpasse de transmitterede data derefter!
Her et enkelt sølle eksempel af mange observerede (ingen særlig grund til at klandre den specifikke afsender her, blev bare trigget af at klikke på et link i et nyhedsbrev just modtaget) fra SDFE‘s nyhedsoversigtsside, hvor det er så åbenlyst tydeligt på billedindlæsningshastigheden at billederne er alt for meget større end nødvendigt.
Inspicering i browseren (f.eks. med Firefox dev-tools: F12->Network) viser da også at billedindholdet fylder 14,85 MiB ud af sidens samlede størrelse på 16,23 MiB (91,5%).
SDFE.dk nyhedsoversigtsside, billederoverførsel
Hvad værre er at billederne på denne side kun anvendes som meget små illustrationer til en række artikler, men hentes som enorme filer i opløsningen 1920×1080. Inspektion af det første billede i browserens DOM-inspektør (Firefox: F12->Inspector) afslører at browseren kun viser billedet i en opløsning på 100×56! Der går altså 1920*1080-100*56=2’068’000 (99,73%) pixels til fuldstændig spilde.
SDFE.dk digebillede, vist størrelse vs. datastørrelse
Et hurtigt forsøg med at nedskalere et enkelt billede (terminal-foo!), uden at gå ind i så avancerede emner som JPG-kompressionskvalitet eller alternative billedformater (for ikke at tale om reel billedoptimering), kan give en ide om hvor meget data der egentlig overføres unødigt ved visning af denne side:
$ wget –quiet https://sdfe.dk/media/2919562/dige-1920×1080.jpg $ gm convert -resize 100×56 dige-1920×1080.jpg dige-100×56.jpg $ ls -l dige-* -rw-rw-r– 1 miki miki 17420 Apr 30 15:54 dige-100×56.jpg -rw-rw-r– 1 miki miki 1568308 Apr 24 14:54 dige-1920×1080.jpg $
Billedets størrelse reduceres her fra 1’568’308 bytes (1,60 MiB) til 17’420 bytes (17,01 KiB). Det er relativt en besparelse i overførselsmængden på dette billede på 8903%, vel at mærke med nøjagtigt det samme visuelle udbytte! For ikke at tale om den besparelse i hukommelsesforbrug og processeringstid der også opås ved at sidebeskuerens browser skal forholde sig til en tilsvarende mindre mængde data.
Under antagelse af at tilsvarende reduktioner kan opnås på det samlede sæt af billeder på denne side, vil overførselsbehovet kunne reduceres fra 15’206,40 KiB til 170,80 KiB. Tænk lige over det næste gang, webmaster!
Rygterne har lydt noget tid, men nu er det officielt at formand for og stifter af Free Software Foundation, den ideologiske ophavsmand til GNU-projektet og højlydt fortaler for softwarebrugeres frihed og privatliv i den digitale verden, Richard Stallman besøger Danmark med en række åbne og gratis foredrag dette forår.
Det er Stallmans dedikerede arbejde med fri software og GNU-projektet fra starten af 1980’erne, herunder udformning af softwarelicenser som GNU GPL og udviklingsværktøjer som GNU Compiler Collection (GCC) og GNU Emacs, der er grundlaget for en stor del af det der i offentligheden i dag bedst kendes som “open source”. I Stallmans og GNUs terminologi benævnes det dog retteligt “fri software” (på engelsk: “free software”) for at fremhæve at etablering og bevarelse af softwarens, og slutbrugeren af dens, frihed er det egentlige rationale for at give kildekoden fri.
Kernen Linux er frigivet under GNU GPL og er både inspireret af og anvender GNU-projektets arbejde direkte, og er en vigtig del af et komplet GNU-system (også kendt som GNU/Linux eller en “Linux-distribution”).
Stallman kommer på en veritabel Danmarksturne med start i Aalborg mandag d. 6. maj 2019 og ender i København fredag d. 10. maj 2019. Foredragsrækken er arrangeret af innovationsnetværket for IT, InfiniIT, som inkluderer de store IT-universiteter i Danmark.
Den samlede foredragsrække er som følger:
16:00-19:00
16:00-18:00
(ændrede lokationsoplysninger ikke opdaterede her)
Kilder: FSF: Where in the world is Richard Stallman?, InfinIT-arrangementer, IDA-søgning, PROSA-arrangementer
Begivenhederne er også tilføjet den åbne kalender GriCal: grical.org/s/?query=%40DK.
De primære kilder er hovedsageligt på engelsk:
I regularly stumble upon maps which I cane recognize as derived from OpenStreetMap. Usually by map style, the level of detail or by seeking out details I know is only present in OSM. I usually inspect the attribution and license accompanying the material to assess whether it is prepared by an organisation who understands the needs to and cares about having its chain of copyright somewhat traceable. I’ll attempt to do a small concise writeup of the findings going along this in the future. Maybe I’ll be able to produce a series of “OSM Use & Attribute” posts?
Quite randomly the first one became the Danish company Bisnode.dk using OSM tiles from Mapquest (the ones who fled from an open approach in 2015) to indicate locations in the commercial and subscription-required yellow pages product named Bisnode Navne & Numre® Erhverv (aka. erhverv.nnmarkedsdata.dk).
For reference: OSM’s Copyright and License page
Below is a screen-shot from within the walled garden, and the actual link leading to the page (accessible to me because I’m the owner of the company in question).
Bisnode obviously uses Mapquest as supplier of their maps. Mapquest do make sure to display a nice and prominent copyright attribution line in the map with text linking to respectively Mapquest’s Mapbox attribution page (“MAPBOX” text), OpenStreetMap’s about page (“OPENSTREETMAP” text) and Mapquest’s TOS (“TERMS” text).
The OSM attribution doesn’t comply exactly with the words of the OSM attribution instructions; to use the text “© OpenStreetMap contributors” and make it clear that data is distributed under the ODbL license (suggested done by linking to the OSM copyright page which only happens indirectly through the about page). This is almost fulfilled on the 3rd Party Notices and Licences which is, however, buried behind the long read of the Mapquest TOS, and where the attribution strangely becomes the rather twisted “© Open Street Map and Contributors”. Who are OSM beyond its contributors? And why doesn’t Mapquest know the correct name of the project? Is it to prevent search engine hits of the more unique concatenation of the three words? That’s a guess.
Strangely an almost similar text but also with a reference to the ODbL is to be found in the “Mapbox Streets” section of the Mapbox attribution page.
For reference: OSM/OSMF’s Tile Usage Policy
The tiles are obviously rendered using the Mapquest cartography and thus can be served only by Mapquest.
