Process CSV #2

Closed
opened 2020-02-17 19:09:03 +00:00 by ardouglass · 0 comments
ardouglass commented 2020-02-17 19:09:03 +00:00 (Migrated from github.com)
  • Upload CSV locally
  • Check CSV for required fields
    • Address
    • City
    • State
    • ZIP
  • Store record of bad rows
  • Batch good rows into groups of 25 - rate limiting
  • Loop through each batch at 1 batch per second and query: https://maps.googleapis.com/maps/api/geocode/json?address=${address}&key=${GOOGLE_MAPS_API_KEY} - Google Maps Docs
  • Take data and results, and put in an array of JSON objects with the below structure:
{
    "id": "d131dd02c5e6eec4", // hash of the content of the CSV row
    "lat": 0.00,
    "lng": 0.00,
    "data": {...} // all the fields of the csv 
}
  • Display bad rows that were not processed
  • Post good data to some endpoint
- [x] Upload CSV locally - [x] Check CSV for required fields - [x] Address - [x] City - [x] State - [x] ZIP - [x] Store record of bad rows - [x] Batch good rows into groups of 25 - [rate limiting](https://developers.google.com/maps/documentation/geocoding/geocoding-strategies#rate-limits) - [x] Loop through each batch at 1 batch per second and query: `https://maps.googleapis.com/maps/api/geocode/json?address=${address}&key=${GOOGLE_MAPS_API_KEY}` - [Google Maps Docs](https://developers.google.com/maps/documentation/geocoding/start) - [x] Take data and results, and put in an array of JSON objects with the below structure: ```js { "id": "d131dd02c5e6eec4", // hash of the content of the CSV row "lat": 0.00, "lng": 0.00, "data": {...} // all the fields of the csv } ``` - [x] Display bad rows that were not processed - [x] Post good data to some endpoint
Commenting is not possible because the repository is archived.
No milestone
No project
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: ardouglass/address-mapper-wp#2
No description provided.