-
Notifications
You must be signed in to change notification settings - Fork 1
/
this-week-in-artificial-intelligence-security-30-of-2016.html
199 lines (176 loc) · 12.2 KB
/
this-week-in-artificial-intelligence-security-30-of-2016.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>This week in Artificial Intelligence & Security - #30 of 2016</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<meta name="author" content="Marina von Steinkirch">
<!-- Le styles -->
<link rel="stylesheet" href="./theme/css/bootstrap.dark.css" type="text/css" />
<style type="text/css">
body {
padding-top: 60px;
padding-bottom: 40px;
}
.tag-1 {
font-size: 13pt;
}
.tag-2 {
font-size: 11pt;
}
.tag-2 {
font-size: 10pt;
}
.tag-4 {
font-size: 8pt;
}
</style>
<link href="./theme/css/bootstrap-responsive.dark.css" rel="stylesheet">
<link href="./theme/css/font-awesome.css" rel="stylesheet">
<link href="./theme/css/pygments.css" rel="stylesheet">
<!-- Le fav and touch icons -->
<link rel="shortcut icon" href="./theme/images/favicon.ico">
<link rel="apple-touch-icon" href="./theme/images/apple-touch-icon.png">
<link rel="apple-touch-icon" sizes="72x72" href="./theme/images/apple-touch-icon-72x72.png">
<link rel="apple-touch-icon" sizes="114x114" href="./theme/images/apple-touch-icon-114x114.png">
<link href="./feeds/all.atom.xml" type="application/atom+xml" rel="alternate" title="chmod +x singularity.sh ATOM Feed" />
</head>
<body>
<div class="navbar navbar-fixed-top">
<div class="navbar-inner">
<div class="container-fluid">
<a class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</a>
<a class="brand" href="./index.html">chmod +x singularity.sh </a>
<div class="nav-collapse">
<ul class="nav">
<li class="divider-vertical"></li>
<ul class="nav pull-right">
<li><a href="./authors.html">About</a></li>
<li><a href="./archives.html"><b>Archives</b></a></li>
<li>
<a href="https://github.com/bt3gl">github
<!--<i class="icon-github-sign icon-large" ></i>-->
</a></li>
<li>
<a href="https://twitter.com/1bt337">
<!--<i class="icon-twitter-sign icon-large"></i> -->
twitter
</a></li>
<li><a href="http://bt3gl.github.io/projects_page/index.html">Bygone Playful Times
</a></li>
</ul>
</ul>
<!--<p class="navbar-text pull-right">Logged in as <a href="#">username</a></p>-->
</div><!--/.nav-collapse -->
</div>
</div>
</div>
<div class="container-fluid">
<div class="row">
<div class="span9" id="content">
<section id="content">
<article>
<header>
<h1>
<a href=""
rel="bookmark"
title="Permalink to This week in Artificial Intelligence & Security - #30 of 2016">
This week in Artificial Intelligence & Security - #30 of 2016
</a>
</h1>
</header>
<div class="entry-content">
<div class="well">
<footer class="post-info">
<abbr class="published" title="2016-07-30T00:00:00">
Sat 30 July 2016 </abbr>
<span class="label"> Category</span>
<a href="./category/ai-ml-security.html"><i class="icon-folder-open"></i>AI & ML, Security</a>
</footer><!-- /.post-info --> </div>
<p><img alt="cyberpunk" height="300px" src="./cyberpunk/2.jpg" width="400px" /></p>
<h2>Papers</h2>
<p><strong> This is a selection of quintessential papers for anyone starting on Deep Learning (Thanks to Joe Zimmerman):</strong></p>
<ul>
<li><a href="https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf">ImageNet Classification with Deep Convolutional Neural Networks (Krizhevsky, <em>et al.</em>, 2014)</a>. AlexNet.</li>
<li><a href="https://arxiv.org/pdf/1409.1556.pdf">Very Deep Convolutional Networks for large-scale image recognition (Simonyan, <em>et al.</em>, 2014)</a>. Image classification.</li>
<li><a href="https://arxiv.org/pdf/1207.0580.pdf">Improving neural networks by preventing co-adaptation of feature detectors</a>. Dropout and regularization.</li>
<li><a href="https://arxiv.org/pdf/1503.03832v3.pdf">FaceNet: A Unified Embedding for Face Recognition and Clustering (Schroff, <em>et al.</em>, 2015)</a>. Metric learning (FaceNet). (Metric learning)</li>
<li><a href="https://arxiv.org/pdf/1512.03385v1.pdf">Deep Residual Learning for Image Recognition (He, <em>et al.</em>, 2015)</a>. Very deep networks (ResNet).</li>
<li><a href="https://arxiv.org/pdf/1409.0473v7.pdf">Neural Machine Translation by jointly learning to align and translate (Bahdanau, <em>et al.</em>, 2015)</a>. RNNs, LSTMs, GRUs - machine translation with alignment.</li>
<li><a href="http://arxiv.org/pdf/1412.5903v5.pdf">Deep structured output learning for unconstrained text recognition (Jaderberg, <em>et al.</em>, 2014)</a>. Text recognition.</li>
<li><a href="https://arxiv.org/pdf/1512.02595v1.pdf">Deep Speech 2: End-to-End Speech Recognition in English and Mandarin (Amodei, <em>et al.</em>, 2015)</a>. Speech recognition (DeepSpeech 2).</li>
<li><a href="http://arxiv.org/pdf/1508.06576v2.pdf">A Neural Algorithm of Artistic Style (Gatys, <em>et al.</em>, 2015)</a>. Artistic style transfer.</li>
<li><a href="http://arxiv.org/pdf/1511.08228v3.pdf">Neural GPUs learn algorithms (Kaiser, <em>et al.</em>, 2015)</a>. A Neural GPUs.</li>
<li><a href="http://people.csail.mit.edu/kalyan/AI2_Paper.pdf">AI2: Training a big data machine to defend (Kalyan, <em>et al.</em>, 2016)</a>.</li>
<li><a href="http://download.tensorflow.org/paper/whitepaper2015.pdf">Tensor Flow Whitepaper, (Abadi, <em>et al.</em>, 2014)</a>.</li>
<li><a href="https://lvdmaaten.github.io/publications/papers/Torchnet_2016.pdf">Torchnet: An Open-Source Platform for (Deep) Learning Research, (Collobert, <em>et al.</em>, 2016)</a>.</li>
</ul>
<h2>News</h2>
<ul>
<li><a href="https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html">Using Keras and Deep Q-Network to Play FlappyBird</a>. Hands-on on Google DeepMind's Deep Q-Network.</li>
<li><a href="http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/">Neural Networks, Manifolds, and Topology</a>. This is a 2-years-old article, but a very well-written high-level explanation of the topology of low-dimensional NNs. "<em>The task of a classification algorithm is fundamentally to separate a bunch of tangled manifolds.</em>"</li>
<li><a href="http://colah.github.io/posts/2015-08-Backprop/">Calculus on Computational Graphs: Backpropagation</a>. Backpropagation explained in a very well-written text.</li>
<li><a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">Understanding LSTM Networks</a>. Another hit :).</li>
<li><a href="http://colah.github.io/posts/2015-01-Visualizing-Representations/">Visualizing Representations: Deep Learning and Human Beings.</a> Another Christopher Olah's great post, now on NN's different layers representations, tanging some philosophic aspects of it.</li>
<li><a href="http://cs.stanford.edu/people/karpathy/cnnembed/">Karpathy's t-SNE visualization of CNN codes.</a> He takes the 50k ILSVRC 2012 validation images, extracts the 4096-dimensional fc7 CNN features using Caffe and then uses Barnes-Hut t-SNE to compute a 2-dimensional embedding that respects the high-dimensional (L2) distances. </li>
<li><a href="https://blogs.nvidia.com/blog/2016/01/12/accelerating-ai-artificial-intelligence-gpus/">NVIDIA's Accelerating AI with GPUs: A New Computing Model</a>.</li>
<li><a href="https://code.facebook.com/posts/580706092103929/lighting-the-way-to-deep-machine-learning/">Torchnet: Lighting the way to deep machine learning</a>. "<em><a href="https://github.com/torchnet/torchnet">Torchnet</a> is different from frameworks such as Caffe, Chainer, TensorFlow, and Theano, in that it does not focus on performing efficient inference and gradient computations in deep networks. Instead, Torchnet provides a framework on top of a deep learning framework that makes rapid experimentation easier.</em>"</li>
</ul>
<h2>Videos</h2>
<ul>
<li><a href="http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44921.pdf">Large-Scale Deep Learning for Intelligent Computer Systems by Jeff Dean</a>.</li>
<li><a href="https://www.youtube.com/watch?v=JeBkUtYvBBM">Prof. Adrian Owen on The Search for Consciousness: detecting awareness in the vegetative state (2015)</a>.</li>
<li><a href="https://www.youtube.com/watch?v=Ics9CjRSMfc">Baidu AI Composer</a>.</li>
</ul>
<h2>Tools</h2>
<ul>
<li><a href="https://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground">Understanding neural networks with TensorFlow Playground</a>. </li>
<li><a href="http://cs.stanford.edu/people/karpathy/convnetjs//demo/classify2d.html">Karpathy's ConvNetJS viz tool.</a></li>
</ul>
<h2>Cool Plots</h2>
<ul>
<li>Remember that the hidden layer learns a representation so that the data is linearly separable, so that's is how you do separate a spiral two-dimensional dataset using Tensorflow playground and Convnetjs:</li>
</ul>
<p>With tanh:</p>
<p><img alt="tahn2" height="300px" src="./tensor_flow_playground/tanh2.png" width="400px" /> <img alt="tahn11" height="300px" src="./tensor_flow_playground/tan1.png" width="400px" /> <img alt="tan2" height="300px" src="./tensor_flow_playground/tan2.png" width="400px" /> <img alt="tan3" height="300px" src="./tensor_flow_playground/tan3.png" width="400px" /> </p>
<p>With ReLU:</p>
<p><img alt="relu2" height="300px" src="./tensor_flow_playground/relu2.png" width="400px" /></p>
<ul>
<li>That's is how you do not separate a spiral two-dimensional dataset:</li>
</ul>
<p><img alt="linear" height="300px" src="./tensor_flow_playground/linear.png" width="400px" /> <img alt="relu_no" height="300px" src="./tensor_flow_playground/relu_no.png" width="400px" /> <img alt="sigmoid" height="300px" src="./tensor_flow_playground/sigmoid.png" width="400px" /> <img alt="relu_no2" height="300px" src="./tensor_flow_playground/relu_no2.png" width="400px" /> <img alt="relu_no3" height="300px" src="./tensor_flow_playground/relu_no3.png" width="400px" /> <img alt="relu_no4" height="300px" src="./tensor_flow_playground/relu_no4.png" width="400px" /> </p>
<p><img alt="cyberpunk" height="600px" src="./draws/11.png" width="900px" /></p>
<p><img alt="cyberpunk" height="600px" src="./draws/12.png" width="900px" /></p>
</div><!-- /.entry-content -->
<div class="comments">
<h2>Comments !</h2>
<div id="disqus_thread"></div>
<script type="text/javascript">
var disqus_identifier = "this-week-in-artificial-intelligence-security-30-of-2016.html";
(function() {
var dsq = document.createElement('script');
dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://bt3gl.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] ||
document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
</div>
</article>
</section>
</div><!--/span-->
</div><!--/row-->
<footer>
<address id="about">
</address><!-- /#about -->
</footer>
</div><!--/.fluid-container-->
<script src="./theme/js/jquery-1.7.2.min.js"></script>
<script src="./theme/js/bootstrap.min.js"></script>
</body>
</html>