-
Notifications
You must be signed in to change notification settings - Fork 0
/
pgdfw-veciot.slide
118 lines (87 loc) · 3.94 KB
/
pgdfw-veciot.slide
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
High Resolution Vector Processing and IOT in PostgreSQL
Jun 6, 2018, Dallas, Texas, USA
John Scott
Founder, SetSpace, Inc, 1998-now
jmscott@setspace.com
June 6, 2018
Dallas, Texas, USA
* Tacoma Bridge Collapse in Low Winds
.image pgdfw-veciot/tacoma-video.jpg
.link https://youtu.be/lXyG68_caV4?t=61 Video of Tacoma Bridge Collapse
* Flexible Structures Vibrate at Peculiar Frequencies
.image pgdfw-veciot/schematic-mode.gif
.caption Small Energy Stress Certain Points, Like a Coat Hanger
* Cycle of Monitoring of Flexible Structure
.image pgdfw-veciot/monitor-cycle.jpg _ 640
* Cheap Inertial Measurement Units (IMU) Changed the World
.image pgdfw-veciot/imu-schematic2.jpeg _ 820
.caption Micro-Electro Mechanics Sensor (MEMS) In Smartphones, FitBit, Drones, Aircraft
* Inside an IMU - Flow of Micro Electronic Mechanical Sensors (MEMS)
.image pgdfw-veciot/imu-schematic.png _ 1024
* Linear Accelometer (MEMS) Viewed in Electron Microscope!
.image pgdfw-veciot/linear-imu.jpg 520 _
* Angular Accelometer (MEMS) Viewed in Electron Microscope!
.image pgdfw-veciot/angular-imu.jpg 520 _
* VectorNav 300 with Gravitational and Magnetic Map of Earth
.image pgdfw-veciot/vnav-300.jpg 480 _
.caption Ruggedized with 2 GPS External Antenna for Recalibrating the Integrated Kalman Filter
* VectorNac 300 Fits in Your Hand - $800
.image pgdfw-veciot/vnav-hand.jpg _ 880
.caption Made in Dallas, Texas!
* Inertial Measurement Units in Everything!
.image pgdfw-veciot/horse-imu.png _ 1000
* Drones has Many IMUs + GPS
.image pgdfw-veciot/drone-imu.png
* Why Monitor Flexible Structures?
- Alert Owner of Dramatic Movement of Bridge
- Twist, Sway and Motion Stress on Structure
- Slow Moving Oscillations - Coat Hanger Bending
- Prioritize Maintence After Extreme Weather
- Detect Permanent Deformations
- Insurance Benefits
* Modal Analysis and Movement Done with Fast Fourier Transform
.image pgdfw-veciot/fft-psd-spectrum.jpg
Important - All Analysis Done on 1024 Sample Vectors !!
* Bread and Butter of Vibrational/Modal Analysis
.image pgdfw-veciot/case-study.jpg _ 960
* Case Study - High Resolution Samples of 10 Vectors/Second/Sensor
- Our Customer is a Market Maker
- Oscillations up to 5 Times/Second (Nyquist Theorem)
- 1.5GB/month per sensor, uncompressed
- At Least Two Sensors per Structure
- Speced to 100,000 sensors -> 145 TB/month, uncompressed
* YMR Samples Arrive from Bridge as a CSV File (via rsync)
.code pgdfw-veciot/csv-ymr.code
.html pgdfw-veciot/csv-ymr.term
.caption 1024 * 3 samples (rows) == 5.1min in single log segment, about 64kbytes
* Why Use PostgreSQL Instead of MongoDB/Spark?
- SQL Best for Complex Historical Patterns in Data
- PostgreSQL Easy to Extend to File Store for Samples (FDW)
- PostgreSQL Allow New Data Types - Vector Can be Hidden in File Store
- Materialized Views of MATLAB Calculations
- A/B Comparison (Profiles) of Motion and Mode Algorithms
We Always Have CitusDB!
* Vector Sample Log Table - bridge_log_vnymr_10hz
- Stores 5.1min @ 10hz == 3072 Tuples
- Key is Surrogate Sensor Id and Start Time for Each of 3, 1024 Segments
- Single PG Vector/Array for Sample Times
- Single PG Vector/Array for 12 Dimensional Floating Point Samples
- Compress Very Well
* SQL for Sample Times in Table bridge_log_vnymr_10hz
.code pgdfw-veciot/sql-sample-time.code
.caption Table bridge_log_vnymr_10hz
* SQL for 1024 * 3 * 12 YMR Samples in Table bridge_log_vnymr_10hz
.code pgdfw-veciot/sql-ymr.code
.caption Table bridge_log_vnymr_10hz
* Vectors Compress Well but Tricky in SQL So Unpack with a View
.html pgdfw-veciot/view-ymr.code
* Unpacked View is Easy to Query
.code pgdfw-veciot/view-ymr-sql.code
* Materialize Views Store MATLAB Results
.html pgdfw-veciot/matlab-sql.code
* Vectors Can Reference Raw CSV Files Using Foreign Data Wrappers
- Use Custom Data Type - Can Not Override array[] (JavaScript has same problem)
- Vectors < 3 Months in PostgreSQL
- Vectors > 3 and < Year in CSV File
- Null Otherwise
* Storing Dense Vector Data Can be Done in PostgreSQL