Project Nominatim, Setup

At the time of writing this article, my docker version is
$ docker --version
Docker v19.0.3 

Quickly steps is to upgrade the Docker-ce
$ sudo apt upgrade docker-ce
$ docker --version
Docker version 23.0.1, build a5ee5b1

Or if you prefer the "workaround" i went thought feel free to follow 
steps bellow.

Here are the step-by-step:
a. Download or git clone project nominatim Dockerfile here
    */ v.4.2 is the selected one at the time this article being written
b. $ docker build -t nominatim .
c. In my PC somehow the apt-get update did not run well,
    it keeps saying that,
      "W: GPG error: http://security.ubuntu.com/ubuntu jammy-security InRelease:
           The following signatures couldn't be verified because the public key is
           not available: NO_PUBKEY 871920D1991BC93C
        E: The repository 'http://security.ubuntu.com/ubuntu jammy-security InRelease'
           is not signed."

Ok, the dockerfile just won't work, the solution is to pass this option in
the docker run syntax:

$ docker run -it --security-opt seccomp:unconfined <and so on...> <and so on...>

But, the option can not be passed during the docker build. Well the automatic
solution is not working, therefor i need to go manual mode and Dockerfile
is a helpful resource to start with.

Starts with looking what are the command and components to be run manually:


I copy-pasted with little modification using Vim:

$ apt-get update

$ apt-get -y install \
        locales \
    && locale-gen en_US.UTF-8 \
    && update-locale LANG=en_US.UTF-8

$ apt-get -y install \
        -o APT::Install-Recommends="false" \
        -o APT::Install-Suggests="false" \
        build-essential \
        g++ \
        cmake \
        libpq-dev \
        zlib1g-dev \
        libbz2-dev \
        libproj-dev \
        libexpat1-dev \
        libboost-dev \
        libboost-system-dev \
        libboost-filesystem-dev \
        liblua5.4-dev \
        postgresql-contrib \
        postgresql-server-dev-14 \
        postgresql-14-postgis-3 \
        postgresql-14-postgis-3-scripts \
        php \
        php-intl \
        php-pgsql \
        php-cgi \
        apache2 \
        libapache2-mod-php \
        python3-dev \
        python3-pip \
        python3-tidylib \
        python3-psycopg2 \
        python3-setuptools \
        python3-dotenv \
        python3-psutil \
        python3-jinja2 \
        python3-datrie \
        python3-icu \
        python3-argparse-manpage \
        git \
        curl \
        sudo

Make sure that the `#` are all taken out. Another thing to keep in-mind that most of the ubuntu
distros has the LANG compatible to en_US.UTF-8. As i had mentioned the update-locale
part does not work in my container therefore i skip it.

Once the apt-get install part had been done, make sure to check the sudo:
$ sudo --help
In my case, somehow sudo did not install properly. Later had to re-exec 
$ apt-get install sudo
Another one is the pip3 install
$ pip3 install osmium 
in the docker container and everything will work as intended. 
At this point you need to create new docker image so that all of 
the process we had would be saved in the new image. Run at host
$ docker ps
$ docker commit <container_id> ubuntu:jammy-unsec2

Backup the original Dockerfile and modify by removing
the apt-get install software part as follow:



Then run the docker build. Once you got it done. 
Login to the container
$ docker exec -it --name ub bash
Check that /var/lib/postgresql has a directory 14/ if yes then copy the directory
to the host. Run this syntax at host,
$ docker cp ub:/var/lib/postgresql/* <your_host_file_path>
In my case is located in /opt/docker/nominatim-docker-master/4.2/postgresql

You are almost arrive at the nominatim import process. Here is the docker-start.sh script.

docker run -it --net=host --rm --name ub --detach --security-opt seccomp:unconfined \
  -e PBF_PATH=/nominatim/data/indonesia-latest-230317.osm.pbf \
  -e NOMINATIM_PASSWORD=123456 \
  -e PROJECT_DIR=/nominatim \
  -e THREADS=4 \
  -p 8080:8080 \
  -v '/opt/docker/nominatim-docker-master/4.2/postgresql:/var/lib/postgresql' \
  -v '/home/adesanto/osmprj:/nominatim/data' \
  nominatim-ubuntu:jammy-unsec4-4.2.1 \
        /bin/bash -c 'cp /nominatim/.env /; chown postgres /var/lib/postgres/* ; /app/start.sh; bash'

You could set NOMINATIM_PASSWORD to anything. And the .env it self if you look-at
the init.sh there is syntax to load the file from root `/.env` directory. All set now file 
the docker-start.sh script. You will see the processes are now running sequentially.
By importing the osm file into postgres with Nominatim. After 3-4 hours of process
and it took 34GB of postgress data. It is time to test:

$ curl "http://127.0.0.1:8080/search?q=gereja+paroki+trinitas&format=xml&polygon_geojson=1&addressdetails=1"

<?xml version="1.0" encoding="UTF-8" ?>
<searchresults timestamp='Wed, 22 Mar 23 20:15:19 +0700' attribution='Data © OpenStreetMap contributors, ODbL 1.0. http://www.openstreetmap.org/copyright' querystring='gereja paroki trinitas' exclude_place_ids='1173942' more_url='/search.php?q=gereja+paroki+trinitas&amp;addressdetails=1&amp;polygon_geojson=1&amp;exclude_place_ids=1173942&amp;format=xml'>
<place place_id='1173942' osm_type='way' osm_id='516542345' place_rank='30' address_rank='30' boundingbox="-6.1470215,-6.1465975,106.7249803,106.7253974" geojson='{"type":"Polygon","coordinates":[[[106.7249803,-6.1469735],[106.7250152,-6.1470001],[106.7253773,-6.1470215],[106.7253974,-6.1466961],[106.7253076,-6.1466055],[106.7250018,-6.1465975],[106.7249803,-6.1469735]]]}' lat='-6.1468348' lon='106.72518853173358' display_name='Gereja Katolik Paroki Trinitas, Jalan Bambu Kuning 9, RW 03, Cengkareng Barat, Cengkareng, Jakarta Barat, Daerah Khusus Ibukota Jakarta, Jawa, 11730, Indonesia' class='amenity' type='place_of_worship' importance='0.30001'>
<amenity>Gereja Katolik Paroki Trinitas</amenity><road>Jalan Bambu Kuning 9</road><city_block>RW 03</city_block><village>Cengkareng Barat</village><suburb>Cengkareng</suburb><city_district>Jakarta Barat</city_district><city>Daerah Khusus Ibukota Jakarta</city><ISO3166-2-lvl4>ID-JK</ISO3166-2-lvl4><region>Jawa</region><ISO3166-2-lvl3>ID-JW</ISO3166-2-lvl3><postcode>11730</postcode><country>Indonesia</country><country_code>id</country_code></place></searchresults>

It works with the search, and now test the reverse part:

$ curl "http://127.0.0.1:8080/reverse?format=jsonv2&lat=-6.146341063551869&lon=106.7255169673749"

{"place_id":1173942,"licence":"Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright","osm_type":"way","osm_id":516542345,"lat":"-6.1468348","lon":"106.72518853173358","place_rank":30,"category":"amenity","type":"place_of_worship","importance":9.99999999995449e-6,"addresstype":"amenity","name":"Gereja Katolik Paroki Trinitas","display_name":"Gereja Katolik Paroki Trinitas, Jalan Bambu Kuning 9, RW 03, Cengkareng Barat, Cengkareng, Jakarta Barat, Daerah Khusus Ibukota Jakarta, Jawa, 11730, Indonesia","address":{"amenity":"Gereja Katolik Paroki Trinitas","road":"Jalan Bambu Kuning 9","city_block":"RW 03","village":"Cengkareng Barat","suburb":"Cengkareng","city_district":"Jakarta Barat","city":"Daerah Khusus Ibukota Jakarta","ISO3166-2-lvl4":"ID-JK","region":"Jawa","ISO3166-2-lvl3":"ID-JW","postcode":"11730","country":"Indonesia","country_code":"id"},"boundingbox":["-6.1470215","-6.1465975","106.7249803","106.7253974"

Works like charm!







Comments