WARNING: as OSM user “mmd” wisely points out (in comment to OSM diary for this post) the sort of thing described here is dangerous to do by hand, and should not be done on the main production instances (there are testing instances for playing around with the API, a little documentation here).
He also points out that the feature packed JOSM editor actually supports continuing a changeset regardless of where it has been initiated. So if you just need to continue working on a changeset (but remember the 1 hour idle timeout) be sure to check out the JSOM upload documentation. Thanks mmd, for being the sane voice ;).
During an editing session the Android OpenStreetMap editor Vespucci crashed on me, which made the mapping UI unusable (objects greyed out and unable to select or edit, had to purge all data to recover functionality). Luckly, I could still navigate the menus, upload changes and opt to not close the changeset. Now, I had long wondered whether the OSM API allowed to continue amending changes to an open changeset from some other editor, so the quest began.
I had the intention of adding a tag representing the name of a building in Esbjerg known as “ISC Huset” (ISC is an engineering consultancy, see wikipedia, and more about the construction), which houses a number of healtcare clinics. The building’s address is Borgergade 70, 6700 Esbjerg (current address node).
This blog post willl attempt to actually add the tag by hand on the command line.
References to OSM objects used:
Option summary for GNU wget used to do HTTP requests below:
- -nv (or –no-verbose): only report errors and basic information
- -O- (or –output-document=-): write output to standard out (stdout)
- –user=<user name>: authenticate as <user name> using HTTP Basic authentication
- –ask-password: ask on command line for HTTP Basic authentication password
- –post-data=”<data>”: initiate a HTTP POST request with <data> as payload
- –server-response (or -S): show full HTTP headers of server response
NOTE: I’ve broken some long output lines replicated below to make it fit the blog, but inserted an escape character (\) before the inserted newline to help copy’n’pasting.
Retrieve changeset metadata
Lets start by looking at the changeset’s metadata.
This can be done by issuing an unauthenticated GET request to the “/api/0.6/changeset/<changeset id>” endpoint.
Note that the ‘changeset’ element has the attribute ‘open=”true”‘ required to be able to modify the changeset. The editor used to create the changeset needs to have done this without explicitly closing it (Vespucci & JOSM closes by default but can be configured not to, iD always closes).
$ wget -nv -O- http://api.openstreetmap.org/api/0.6/changeset/88490797
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.8.3 (802 thorn-02.openstreetmap.org)" copyright="OpenStreetMap and contributors"\
attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
<changeset id="88490797" created_at="2020-07-25T09:51:30Z" open="true" user="mikini" uid="2051"\
min_lat="55.4654072" min_lon="8.4378053" max_lat="55.5258532" max_lon="8.4639026" comments_count="0" changes_count="68">
<tag k="source" v="survey;research"/>
<tag k="locale" v="da-DK"/>
<tag k="created_by" v="Vespucci 14.1.4.0"/>
<tag k="comment" v="Details at Klevestien & Borgergade in Esbjerg and Tarp."/>
</changeset>
</osm>
2020-07-25 14:26:52 URL:http://api.openstreetmap.org/api/0.6/changeset/88490797 [709] -> "-" [1]
$
Pinging the changeset with an empty osmChange structure
To test that the changeset is open, and that we can authenticate correctly, lets try amending it with an empty osmChange structure.
This can be done by issuing an authenticated POST request to the “/api/0.6/changeset/<changeset id>/upload” endpoint.
This also seem to reset the 60 minute timer used for auto-closing the changeset (see mention of “idle timeout” in changeset wiki article).
$ wget -nv -O- --user=mikini --ask-password https://api.openstreetmap.org/api/0.6/changeset/88490797/upload --post-data="<osmChange></osmChange>"
Password for user ‘mikini’:
Authentication selected: Basic realm="Web Password", charset="UTF-8"
<?xml version="1.0" encoding="UTF-8"?>
<diffResult version="0.6" generator="CGImap 0.8.3 (8531 thorn-03.openstreetmap.org)" copyright="OpenStreetMap and contributors"\
attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/"/>
2020-07-25 14:27:20 URL:https://api.openstreetmap.org/api/0.6/changeset/88490797/upload [278] -> "-" [1]
$
Procedure to change the building
The API’s “changeset/<changeset id>/upload” method supports only modifications encoded in the osmChange format which requires changes to be described as complete way/node/relation objects. That is, you can not ask the API to “add this tag to this way”, you need instead to describe the modified way completely saying “this way now looks like this”, including anything (like node references or existing tags) that was not modified. So to make a modificatino to the building’s way we need to retrieve the current version, modify it as needed and upload the complete new way.
Thus the procedure contains these steps;
- Retrieve current building (GET /way/*)
- Modify building data (locally)
- Amend changeset (POST /changeset/*/upload)
Retrieving current building (GET /way/*)
This can be done by issuing an unauthenticated GET request to the “/api/0.6/way/<way id>” endpoint.
The stdin splitter ‘tee’ is used here to both show the result in terminal and put it into file 185369466_v3.osc that we can use for amending the way with the wanted modifications.
$ wget -nv -O- http://api.openstreetmap.org/api/0.6/way/185369466|tee 185369466_v3.osc
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.8.3 (28697 thorn-01.openstreetmap.org)" copyright="OpenStreetMap and contributors"\
attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
<way id="185369466" visible="true" version="3" changeset="84400254" timestamp="2020-04-30T09:35:21Z" user="mikini" uid="2051">
<nd ref="1959614623"/>
<nd ref="1959614727"/>
<nd ref="6299449794"/>
<nd ref="1959614650"/>
<nd ref="1959614630"/>
<nd ref="6299449793"/>
<nd ref="1959614765"/>
<nd ref="7466482063"/>
<nd ref="7466482064"/>
<nd ref="7466482065"/>
<nd ref="7466482062"/>
<nd ref="1959614729"/>
<nd ref="1959614623"/>
<tag k="building" v="yes"/>
</way>
</osm>
2020-07-25 14:54:30 URL:http://api.openstreetmap.org/api/0.6/way/185369466 [769] -> "-" [1]
$
Modify building data (locally)
Now we need to build an osmChange file out of the existing <way>…</way> element from the output above describing the wanted building. This involves;
- removing the <xml> tag (maybe not needed, but it is not mentioned at all in the osmChange documentation)
- replacing <osm> with <osmChange>
- adding a <modify> element to the <osmChange> element, containing the existing <way>
- update the “changeset” attribute to the changeset we’re amending
- amending the contents of <way> with the wanted <tag>s
Use your favorite editor (emacs would be my preference) to load the 185369466_v3.osm file, make the modifications and save it as 185369466_v4.osc. OSM tags are a XML empty-element tags containing the OSM tag’s key and value in the “k” and “v” attributes, thus the “name” tag of the building I needed to add would be ‘<tag k=”name” v=”ISC Huset”/>’, I also added some other related tags (“source:name” and “website”).
The finished .osc file now looks like this;
$ cat 185369466_v4.osc
<osmChange>
<modify>
<way id="185369466" visible="true" version="3" changeset="88490797" timestamp="2020-04-30T09:35:21Z" user="mikini" uid="2051">
<nd ref="1959614623"/>
<nd ref="1959614727"/>
<nd ref="6299449794"/>
<nd ref="1959614650"/>
<nd ref="1959614630"/>
<nd ref="6299449793"/>
<nd ref="1959614765"/>
<nd ref="7466482063"/>
<nd ref="7466482064"/>
<nd ref="7466482065"/>
<nd ref="7466482062"/>
<nd ref="1959614729"/>
<nd ref="1959614623"/>
<tag k="building" v="yes"/>
<tag k="name" v="ISC Huset"/>
<tag k="source:name" v="sign;website"/>
<tag k="website" v="https://www.isc.dk/isc-huset-esbjerg/"/>
</way>
</modify>
</osmChange>
$
Wdiff’ing against the .osm source shows exactly what changed (additions between “{+” & “+}”, removals between “[-” & “-]”);
$ wdiff 185369466_v3.osm 185369466_v4.osc
[-<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.8.3 (28697 thorn-01.openstreetmap.org)" copyright="OpenStreetMap and contributors"\
attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">-]{+<osmChange>
<modify>+}
<way id="185369466" visible="true" version="3" [-changeset="84400254"-] {+changeset="88490797"+} timestamp="2020-04-30T09:35:21Z" user="mikini" uid="2051">
<nd ref="1959614623"/>
<nd ref="1959614727"/>
<nd ref="6299449794"/>
<nd ref="1959614650"/>
<nd ref="1959614630"/>
<nd ref="6299449793"/>
<nd ref="1959614765"/>
<nd ref="7466482063"/>
<nd ref="7466482064"/>
<nd ref="7466482065"/>
<nd ref="7466482062"/>
<nd ref="1959614729"/>
<nd ref="1959614623"/>
<tag k="building" v="yes"/>
{+<tag k="name" v="ISC Huset"/>
<tag k="source:name" v="sign;website"/>
<tag k="website" v="https://www.isc.dk/isc-huset-esbjerg/"/>+}
</way>
[-</osm>-]
{+</modify>
</osmChange>+}
$
Amend changeset (POST /changeset/*/upload)
Now, we’ll again use the changeset upload method but this time supplying our actual osmChange elemet in the .osc file.
The output is a bit elaborate, because I had enabled full output from wget while debugging what changes to the <way> element was needed for the server to accept the upload (only the “changeset” attribute needs to match the open changeset as outlined in the “Modify building data” above). I’ve highligted the actual server response telling that the changes were accepted and way #185369466 is now at v4.
$ wget -S -O- --user=mikini --ask-password https://api.openstreetmap.org/api/0.6/changeset/88490797/upload --post-file=185369466_v4.osc
Password for user ‘mikini’:
--2020-07-25 15:44:41-- https://api.openstreetmap.org/api/0.6/changeset/88490797/upload
Resolving api.openstreetmap.org (api.openstreetmap.org)... 130.117.76.12, 130.117.76.13, 130.117.76.11, ...
Connecting to api.openstreetmap.org (api.openstreetmap.org)|130.117.76.12|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.1 401 Unauthorized
Date: Sat, 25 Jul 2020 13:44:41 GMT
Server: Apache/2.4.29 (Ubuntu)
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly"
WWW-Authenticate: Basic realm="Web Password", charset="UTF-8"
Cache-Control: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly"
Content-Length: 22
Content-Type: text/plain; charset=utf-8
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Authentication selected: Basic realm="Web Password", charset="UTF-8"
Reusing existing connection to api.openstreetmap.org:443.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Sat, 25 Jul 2020 13:44:42 GMT
Server: Apache/2.4.29 (Ubuntu)
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly"
Content-Encoding: identity
Cache-Control: private, max-age=0, must-revalidate
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly"
Vary: Accept-Encoding
Content-Type: application/xml; charset=utf-8
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
Transfer-Encoding: chunked
Length: unspecified [application/xml]
Saving to: ‘STDOUT’
- [<=> ] 0 --.-KB/s
<?xml version="1.0" encoding="UTF-8"?>
<diffResult version="0.6" generator="CGImap 0.8.3 (8537 thorn-03.openstreetmap.org)" copyright="OpenStreetMap and contributors"\
attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
<way old_id="185369466" new_id="185369466" new_version="4"/>
</diffResult>
- [ <=> ] 353 --.-KB/s in 0s
2020-07-25 15:44:42 (20,3 MB/s) - written to stdout [353]
$
Result
That was it, the building is now named in OSM, in a changset amended by hand.
Take a look at https://www.openstreetmap.org/way/185369466/.
