maxmind legacy data file no longer available for Insights

Description

While updating Insights to the ironwood.master release, I also wanted to update the maxmind data file (country granularity) to be uploaded to HDFS or S3 for the geolocation task.

The Legacy maxmind data file have been discontinued since January 2, 2019 as per https://support.maxmind.com/geolite-legacy-discontinuation-notice/

The new format is GeoLite2. Unfortunately, according to the same page, "However, you will need to update your GeoLite Legacy integrations to work with However, you will need to update your GeoLite Legacy integrations to work with GeoLite2 databases."

Therefore Insights must be updated to use GeoLite2 databases.

Steps to Reproduce

According to https://edx-analytics-pipeline-reference.readthedocs.io/en/latest/running_tasks.html you should use the following to get the maxmind geo data file.

wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz

This now returns an error message.

wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
--2019-07-02 09:22:29-- http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
Resolving geolite.maxmind.com (geolite.maxmind.com)... 104.17.201.89, 104.17.200.89
Connecting to geolite.maxmind.com (geolite.maxmind.com)|104.17.201.89|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-07-02 09:22:29 ERROR 404: Not Found.

See also

It would be useful if either Open edX moves to use GeoIP2 data files or distributes a GeoIP legacy file available for its Open Source instances. The best solution would be to convert to GeoIP2 for increased accuracy of the geolocation data returned by Insights.

Please take note that Insights uses https://pypi.org/project/pygeoip/ and that GeoIP2 files would require the use of https://pypi.org/project/geoip2/

I don't think it is a straightforward replacement.

Assignee

Unassigned

Reporter

Pierre Mailhot

Labels

None

Platform Area

None

Priority

Unset
Configure