Recommendation: Mark Id constructor as public

[jhimes144]

One of the primary benefits of snowflakes is the ability to query the database for ids that were created after a certain timeframe, without relying on a separate DateTime column. This is how discord does their queries, as 64bit comparisons are faster than date time comparisons. I saw this issue that was closed #60 and I partially disagree with the answer.

We need to be able to create an ID of which represents a point of time, to generate a query to get all items past that moment.

[RobThree]

I think a CreateId(DateTimeOffset dateTime) overload would be a better option as it will at least be able to guarantee the generator-id is a 'known/existing' one. But I'll have to sit on this for a bit to think of any complications that may bring.

[jhimes144]

That sounds good! Great library by the way, thank you so much for making it. Its written well. We ended up forking it for our needs.

[RobThree]

Out of curiosity: what changes did you make? Or was it "just" making the constructor public?

[jhimes144]

I created a method in the ID generator to create a id from a DateTimeOffset, I did not end up making any changes to the ID, to keep its immutable nature.

[RobThree]

The problem I'm facing is: I can't use CreateIdImpl without major refactoring because all clock-handling (clock going backwards etc.) is going out the window with this method. And I don't even know what to do with the sequence; I could take it as a second argument but that would make the user responsible for generating the sequence part. All that's left for the IdGenerator to do then, is set the generator-bits and out the datetime/sequence bits into place.

Also, passing a DateTimeOffset wouldn't work, we'd need an ITimeSource to be passed because we don't know what the tick duration is (can be anything). And that, in turn, would make invoking CreateId a lot more complicated because you'd need to first create an ITimeSource, set it to some date in the past, determine the sequence you desire and then call CreateId with the ITimeSource and generated sequence as arguments.

It's not impossible but overly complicated I think. However. a 'workaround' could be creating another IdGenerator instance with an ITimeSource you can set to your desired date. Normally I don't recommend having more than one IdGenerator in a process/task/thread/whatever and recommend a singleton. But I could see a usecase in this particular case. You could set the date of your custom ITimeSource, generate a bunch of ID's and dispose the IdGenerator. Only issue with this is that you must create Id's chronologically and can't generate them for random times because that will trigger the "Clock moved backwards" mechanism. Or you'd have to instantiate a new IdGenerator for each )non-chronological) Id...

Thoughts are welcome.

[jhimes144]

I hear you, with the way everything is structured it sounds like it's going to be hard to implement in a succinct way. It worked for me because I got rid of all supporting classes and hard coded everything for my use case.

I also needed this to query some data, here is a simple way to convert a DateTimeOffset to the first ID of that moment, ignoring the generatorId (because I want to get all results):

Update: created 2 extension methods to generate the first and the last id, ignoring specific generator and sequence ids (so first id has only 0s after the timestamp part, the last only 1s):

public static long GetFirstIdForDateTime(this IdGenerator generator, DateTimeOffset dateTimeOffset)
{
    var options = generator.Options;
    var ticks = (dateTimeOffset.Ticks - options.TimeSource.Epoch.Ticks) / options.TimeSource.TickDuration.Ticks;
    var timestamp = ticks & (1L << options.IdStructure.TimestampBits) - 1;

    return timestamp << (options.IdStructure.GeneratorIdBits + options.IdStructure.SequenceBits);
}

public static long GetLastIdForDateTime(this IdGenerator generator, DateTimeOffset dateTimeOffset)
{
    var options = generator.Options;
    var ticks = (dateTimeOffset.Ticks - options.TimeSource.Epoch.Ticks) / options.TimeSource.TickDuration.Ticks;
    var timestamp = ticks & (1L << options.IdStructure.TimestampBits) - 1;

    return (timestamp << (options.IdStructure.GeneratorIdBits + options.IdStructure.SequenceBits)) | ((1L << (64 - options.IdStructure.TimestampBits)) - 1);
}

Using this you can create a query to find all the records between 2 dates.

[Rikarin]

This is maybe not so relevant for production use but the Stopwatch has some issues on UNIX when laptop goes to sleep mode. dotnet/runtime#77945

AFAIK the DefaultTimeSource uses Stopwatch internally. When I woke up my laptop without restarting the app, the difference between DateTime.UtcNow and the DateTimeOffset from the ID was in minutes.