My biggest problem was ingesting data. The CSV file I needed to process was above 10MB and unable to be divided using range headers in my requests. This file size was too large to be requested by App Engine. I had to write a separate app on a VPS to process the raw file into a smaller and smaller more efficient file and then ingest that file to app engine.
At the end of last year, App Engine lifted this file size restriction and have been able to adapt the code and now it is full automated.
Comments
I built http://www.scoratic.com/ Its a daily list of the most valuable dropping domain names.
My biggest problem was ingesting data. The CSV file I needed to process was above 10MB and unable to be divided using range headers in my requests. This file size was too large to be requested by App Engine. I had to write a separate app on a VPS to process the raw file into a smaller and smaller more efficient file and then ingest that file to app engine.
At the end of last year, App Engine lifted this file size restriction and have been able to adapt the code and now it is full automated.