Conversation
|
This is to the point where it could use some proper review. I had a lot of lessons-learned along the way. |
dtkav
force-pushed
the
files
branch
8 times, most recently
from
522d143 to
65667f2
Compare
|
I just noticed the files api uses |
| elif isinstance(o, datetime.timedelta): | ||
| return duration_iso_string(o) | ||
| elif isinstance(o, (decimal.Decimal, uuid.UUID, Promise)): | ||
| return str(o) |
There was a problem hiding this comment.
str of a Promise doesn't sound right?
Also Promise doesn't seem to be defined anywhere?
| return data | ||
|
|
||
|
|
||
| class ISODateTimeField(models.DateTimeField): |
There was a problem hiding this comment.
I feel there must be a better way to do this than copy paste this class across django apps.
| data = {} | ||
| for f in chain(opts.concrete_fields, opts.private_fields, opts.many_to_many): | ||
| if f.name is 'id': | ||
| if f.name == 'id': |
| /datalake/admin/whats/: | ||
| get: | ||
| security: | ||
| - jwt: ['admins'] |
There was a problem hiding this comment.
I'm unclear what this part of the tag will do?
What defines who an 'admin' is?
There was a problem hiding this comment.
admins doesn't appear to be defined anywhere else?
2 participants
This API relies on my fork of datalake-common library (datalake-common-dtkav).
planetlabs/datalake-common@master...dtkav:dist
The goal of the Files API is to:
Whatmodel.Design:
DatalakeFile objects are just metadata, and contains a pointer to the content-id (cid) of a file.
The content-id is:
mutibase('base32', mutlihash('blake2b-16', blake2b(file_contents, digest_size=16)))The
work-idfield can be any string (so users can include their own prefixes).However, there are a special set of
work-idprefixes that correspond to objects in mission control.Adding files with these prefixes will make the files available from those objects directly.
A work-id looks like this:
mc-<model_name>.<uuid>.Alternately, a user might be running jobs with jenkins and use something like:
jenkins-job.<id>. These files would not be related to any objects in mission control, but users in the ecosystem can still leverage the datalake to store all of the files in one place.