Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeseries: Add notebook about using CrateDB Cloud with Datashader #186

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/workflows/test-timeseries.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: "Timeseries » Explore"

on:
pull_request:
branches: ~
paths:
- '.github/workflows/test-timeseries.yml'
- 'topic/timeseries/explore/**'
- 'requirements.txt'
push:
branches: [ main ]
paths:
- '.github/workflows/test-timeseries.yml'
- 'topic/timeseries/explore/**'
- 'requirements.txt'

# Allow job to be triggered manually.
workflow_dispatch:

# Run job each night after CrateDB nightly has been published.
schedule:
- cron: '0 3 * * *'

# Cancel in-progress jobs when pushing to the same branch.
concurrency:
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.ref }}

jobs:
test:
name: "
Python: ${{ matrix.python-version }}
CrateDB: ${{ matrix.cratedb-version }}
on ${{ matrix.os }}"
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ 'ubuntu-latest' ]
python-version: [ '3.11' ]
cratedb-version: [ 'nightly' ]

services:
cratedb:
image: crate/crate:nightly
ports:
- 4200:4200
- 5432:5432

steps:

- name: Acquire sources
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
architecture: x64
cache: 'pip'
cache-dependency-path: |
requirements.txt
topic/timeseries/explore/requirements.txt
topic/timeseries/explore/requirements-dev.txt

- name: Install utilities
run: |
pip install -r requirements.txt

- name: Validate topic/timeseries/explore
run: |
ngr test --accept-no-venv topic/timeseries/explore
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
.env
.idea
.pytest_cache
.venv*
__pycache__
.coverage
coverage.xml
mlruns/
archive/
logs.log
logs.log
28 changes: 22 additions & 6 deletions topic/timeseries/explore/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# Time Series with CrateDB
# CrateDB Time Series Exploration and Visualization

This folder provides examples, tutorials and runnable code on how to use CrateDB
for time series use cases.
This folder provides examples, tutorials and runnable code on how to use
[CrateDB] for time series use cases, together with software packages from
the [PyViz] ecosystem, to analyze and explore data on behalf of visually
stunning data plots.

The tutorials and examples focus on being easy to understand and use. They
are a good starting point for your own projects.
Expand All @@ -17,13 +19,27 @@ This folder provides guidelines and runnable code to get started with time serie
walkthrough about how to get started with time series and CrateDB,
and guides you to corresponding example programs.

- [requirements.txt](requirements.txt): For pulling the required dependencies to
run the example programs.

- `cloud-datashader.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](cloud-datashader.ipynb) [![Open in Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/timeseries/explore/cloud-datashader.ipynb)

This notebook explores the [HoloViews] and [Datashader] frameworks and outlines
how to use them to plot the venerable NYC Taxi dataset, after importing it into
a CrateDB Cloud database cluster.

- `timeseries-queries-and-visualization.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](timeseries-queries-and-visualization.ipynb) [![Open in Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/timeseries/explore/timeseries-queries-and-visualization.ipynb)

This notebook explores how to access timeseries data from CrateDB via SQL,
load it into pandas data frames, and visulaize it via plotly.
load it into pandas DataFrames, and visualize it using [Plotly].

It also demonstrates more advanced time series queries in SQL, e.g. aggregations,
It also demonstrates advanced time series queries in SQL, e.g. aggregations,
window functions, interpolation of missing data, common table expressions,
moving averages, JOINs and the handling of JSON data.
moving averages, JOINs, and document store details about handling of JSON data.


[CrateDB]: https://github.com/crate/crate
[Datashader]: https://datashader.org/
[HoloViews]: https://www.holoviews.org/
[Plotly]: https://plotly.com/python/
[PyViz]: https://pyviz.org/
Loading
Loading