AppEngine back-references and timezones bug
Ok, I've learned two important lessons in fuddling around with AppEngine today.
First, there's a bug in the SDK that affects DateTimeProperty and DateProperty objects (since they're all stored in bigtable as datetime.datetime objects, they're really one and the same underneath). Whenever I would type dates in, they were getting returned to me as the day before. Extremely frustrating.
Turns out it's a bug in the SDK. The datastore stores all times as UTC times, so it will translate from your local time to UTC to store the data. Then when it retrieves the date, it should add back in the time zone. This works perfectly in the live build, but the SDK actually subtracts your time zone twice, thus effectively messing up your dates and datetimes. The problem is discussed here along with a successful fix. It involves changing some source code in the appengine folder, but you don't have to recompile or anything.
Second, I've been freaking out trying to get some back-referencing to work out. It's been hard for me to let go of normalized data in the datastore, but I'm warming up to it. One trick is that I'm overriding the save methods for my models so that they update related models as well. That way even though the data is in multiple places, it gets synced up on any kind of save. This is fine and dandy, except sometimes I need to use back-references to make this happen.
Google says to do this using the
First, there's a bug in the SDK that affects DateTimeProperty and DateProperty objects (since they're all stored in bigtable as datetime.datetime objects, they're really one and the same underneath). Whenever I would type dates in, they were getting returned to me as the day before. Extremely frustrating.
Turns out it's a bug in the SDK. The datastore stores all times as UTC times, so it will translate from your local time to UTC to store the data. Then when it retrieves the date, it should add back in the time zone. This works perfectly in the live build, but the SDK actually subtracts your time zone twice, thus effectively messing up your dates and datetimes. The problem is discussed here along with a successful fix. It involves changing some source code in the appengine folder, but you don't have to recompile or anything.
I tried a patch to module datastore_types.py, which seems to work:
change line 1033
from
lambda val: datetime.datetime.fromtimestamp(float(val) / 1000000.0),
to
lambda val: datetime.datetime.utcfromtimestamp(float(val) / 1000000.0),
Per the Python doc, fromtimestamp() converts to the local timezone.
Second, I've been freaking out trying to get some back-referencing to work out. It's been hard for me to let go of normalized data in the datastore, but I'm warming up to it. One trick is that I'm overriding the save methods for my models so that they update related models as well. That way even though the data is in multiple places, it gets synced up on any kind of save. This is fine and dandy, except sometimes I need to use back-references to make this happen.
Google says to do this using the
modelname_setproperty on the model instance. However, you'll get the following TypeError:
'_ReverseReferenceProperty' object is not iterableif you try to retrieve back-references on an object that hasn't been saved yet. There are two ways around this. You can either first save the object,
super(ObjectType, self).save()and then iterate over the back-references, or you can recognize that a new instance won't have and back-references, and check only update back-references if
self.is_saved()is true.
Comments