Skip to content

Commit

Permalink
Finalize Snowflake API for 1.0.0
Browse files Browse the repository at this point in the history
  • Loading branch information
weixiyen committed Feb 17, 2017
1 parent 70d5e90 commit 0e1d648
Show file tree
Hide file tree
Showing 6 changed files with 106 additions and 101 deletions.
4 changes: 4 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## v 1.0.0

- Updated API, swapped Util and Helper module names to be more standard

## v 0.0.2

- added Helper.real_timestamp_of_id
Expand Down
2 changes: 1 addition & 1 deletion lib/snowflake.ex
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ defmodule Snowflake do
import Supervisor.Spec

children = [
worker(Snowflake.Generator, [Snowflake.Utils.epoch(), Snowflake.Utils.machine_id()])
worker(Snowflake.Generator, [Snowflake.Helper.epoch(), Snowflake.Helper.machine_id()])
]

Supervisor.start_link(children, strategy: :one_for_one)
Expand Down
74 changes: 43 additions & 31 deletions lib/snowflake/helper.ex
Original file line number Diff line number Diff line change
@@ -1,50 +1,62 @@
defmodule Snowflake.Helper do
@moduledoc """
The Helper module helps users work with snowflake IDs.
Helper module can do the following:
- Deriving timestamp based on ID
- Creating buckets based on days since epoch...
Utility functions intended for Snowflake application.
epoch() and machine_id() are useful for inspecting in production.
"""
use Bitwise
@default_config [
nodes: [],
epoch: 0
]

@doc """
Get timestamp in ms from your config epoch from any snowflake ID
Grabs epoch from config value
"""
@spec timestamp_of_id(integer) :: integer
def timestamp_of_id(id) do
id >>> 22
@spec epoch() :: integer
def epoch() do
Application.get_env(:snowflake, :epoch) || @default_config[:epoch]
end

@doc """
Get timestamp from computer epoch - January 1, 1970, Midnight
Grabs hostname, fqdn, and ip addresses, then compares that list to the nodes
config to find the intersection.
"""
@spec real_timestamp_of_id(integer) :: integer
def real_timestamp_of_id(id) do
timestamp_of_id(id) + Snowflake.Utils.epoch()
@spec machine_id() :: integer
def machine_id() do
nodes = Application.get_env(:snowflake, :nodes) || @default_config[:nodes]
host_addrs = [hostname(), fqdn(), Node.self()] ++ ip_addrs()

case MapSet.intersection(MapSet.new(host_addrs), MapSet.new(nodes)) |> Enum.take(1) do
[matching_node] -> Enum.find_index(nodes, fn node -> node == matching_node end)
_ -> 1023
end
end

@doc """
Get bucket value based on segments of N days
"""
@spec bucket(integer, atom, integer) :: integer
def bucket(units, unit_type, id) do
round(timestamp_of_id(id) / bucket_size(unit_type, units))
defp ip_addrs() do
case :inet.getifaddrs do
{:ok, ifaddrs} ->
ifaddrs
|> Enum.flat_map(fn {_, kwlist} ->
kwlist |> Enum.filter(fn {type, _} -> type == :addr end)
end)
|> Enum.filter_map(fn {_, addr} -> tuple_size(addr) in [4, 6] end, fn {_, addr} ->
case addr do
{a, b, c, d} -> [a, b, c, d] |> Enum.join(".") # ipv4
{a, b, c, d, e, f} -> [a, b, c, d, e, f] |> Enum.join(":") # ipv6
end
end)
_ -> []
end
end

@doc """
When no id is provided, we generate a bucket for the current time
"""
@spec bucket(integer, atom) :: integer
def bucket(units, unit_type) do
timestamp = System.os_time(:milliseconds) - Snowflake.Utils.epoch()
round(timestamp / bucket_size(unit_type, units))
defp hostname() do
{:ok, name} = :inet.gethostname()
to_string(name)
end

defp bucket_size(unit_type, units) do
case unit_type do
:hours -> 1000 * 60 * 60 * units
_ -> 1000 * 60 * 60 * 24 * units # days is default
defp fqdn() do
case :inet.get_rc[:domain] do
nil -> nil
domain -> hostname() <> "." <> to_string(domain)
end
end
end
51 changes: 51 additions & 0 deletions lib/snowflake/util.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
defmodule Snowflake.Util do
@moduledoc """
The Util module helps users work with snowflake IDs.
Util module can do the following:
- Deriving timestamp based on ID
- Creating buckets based on days since epoch...
- get real timestamp of any ID
"""
use Bitwise

@doc """
Get timestamp in ms from your config epoch from any snowflake ID
"""
@spec timestamp_of_id(integer) :: integer
def timestamp_of_id(id) do
id >>> 22
end

@doc """
Get timestamp from computer epoch - January 1, 1970, Midnight
"""
@spec real_timestamp_of_id(integer) :: integer
def real_timestamp_of_id(id) do
timestamp_of_id(id) + Snowflake.Helper.epoch()
end

@doc """
Get bucket value based on segments of N days
"""
@spec bucket(integer, atom, integer) :: integer
def bucket(units, unit_type, id) do
round(timestamp_of_id(id) / bucket_size(unit_type, units))
end

@doc """
When no id is provided, we generate a bucket for the current time
"""
@spec bucket(integer, atom) :: integer
def bucket(units, unit_type) do
timestamp = System.os_time(:milliseconds) - Snowflake.Helper.epoch()
round(timestamp / bucket_size(unit_type, units))
end

defp bucket_size(unit_type, units) do
case unit_type do
:hours -> 1000 * 60 * 60 * units
_ -> 1000 * 60 * 60 * 24 * units # days is default
end
end
end
62 changes: 0 additions & 62 deletions lib/snowflake/utils.ex

This file was deleted.

14 changes: 7 additions & 7 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ In your mix.exs file:

```elixir
def deps do
[{:snowflake, "~> 0.0.1"}]
[{:snowflake, "~> 1.0.0"}]
end
```

Expand All @@ -26,7 +26,7 @@ Specify the nodes in your config. If you're running a cluster, specify all the

```elixir
config :snowflake,
nodes: [127.0.0.1], # up to 1023 nodes
nodes: ["127.0.0.1"], # up to 1023 nodes
epoch: 1142974214000 # don't change after you decide what your epoch is
```

Expand All @@ -37,34 +37,34 @@ Snowflake.next_id()
# => {:ok, 54974240033603584}
```

## Helper functions
## Util functions

After generating snowflake IDs, you may want to use them to do other things.
For example, deriving a bucket number from a snowflake to use as part of a
composite key in Cassandra in the attempt to limit partition size.

Lets say we want to know the current bucket for an ID that would be generated right now:
```elixir
Snowflake.Helper.bucket(30, :days)
Snowflake.Util.bucket(30, :days)
# => 5
```

Or if we want to know which bucket a snowflake ID should belong to, given we are
bucketing by every 30 days.
```elixir
Snowflake.Helper.bucket(30, :days, 54974240033603584)
Snowflake.Util.bucket(30, :days, 54974240033603584)
# => 5
```

Or if we want to know how many ms elapsed from epoch
```elixir
Snowflake.Helper.timestamp_of_id(54974240033603584)
Snowflake.Util.timestamp_of_id(54974240033603584)
# => 197588482172
```

Or if we want to know how many ms elapsed from computer epoch (January 1, 1970 midnight). We can use this to derive an actual calendar date.
```elixir
Snowflake.Helper.real_timestamp_of_id(54974240033603584)
Snowflake.Util.real_timestamp_of_id(54974240033603584)
# => 1486669389497
```

Expand Down

0 comments on commit 0e1d648

Please sign in to comment.