TensorFlow: Advanced Techniques/C2 Week 1 - Custom and Distributed Training with TensorFlow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# HEADLINE\n",
    "\n",
    "### *Week 1*\n",
    "\n",
    "#### *DeepLearning.AI*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Description: Differentiation and Gradients\n",
    "\n",
    "This week, you will get a detailed look at the fundamental building blocks of TensorFlow - tensor objects. For example, you will be able to describe the difference between eager mode and graph mode in TensorFlow, and explain why eager mode is very user friendly for you as a developer. You will also use TensorFlow tools to calculate gradients so that you don’t have to look for your old calculus textbooks next time you need to get a gradient!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor Basics\n",
    "\n",
    "### What is tensor?\n",
    "\n",
    "![tensor](tensor-revisited.png)\n",
    "\n",
    "#### Some types of tensors\n",
    "\n",
    "**Variable**: `tf.Variable(\"hello\", tf.string)`\n",
    "\n",
    "**Constants**: `tf.constant([1, 2, 3, 4, 5, 6])`\n",
    "\n",
    ">when using a tf.Variable, you're tensor can be modified, which means that if it stores the value \"hello\", this can be changed to store, say the value \"goodbye\". When a variable's values can be modified, we say that it's mutable. Mutable is just the software engineering way of saying that an object can be changed.\n",
    "\n",
    "> But when using tf.constant, it is immutable, which means that it cannot be changed. If a tf.constant held the values one, two, three, four, five, six, you couldn't replace the one with a zero or append another number seven at the end of this tensor, you'd have to create a new tf.constant if you wanted to store a different set of values. \n",
    "\n",
    "![Tensor-character](tensor-character.png)\n",
    "\n",
    "`tf.Tensor([4 6], shape(2,), dtype=int32)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating tensors in code\n",
    "\n",
    "```Python\n",
    "model = tf.keras.Sequential([\n",
    "    tf.keras.layers.Dense(1, input_shape=(1, ))\n",
    "])\n",
    "\n",
    "model.variables\n",
    "```\n",
    "\n",
    "**Creating Tensors with tf.Variable**\n",
    "\n",
    "`vector = tf.Variable(initial_value = [1, 2], dtype=tf.float32)`\n",
    "\n",
    "The shape of the tensor is derived from the initial value.\n",
    "\n",
    "`vector = tf.Variable([1,2,3,4], shape=(2,2)) # occur error`\n",
    "\n",
    "`vector = tf.Variable([1,2,3,4,5], shape=tf.TensorShape(None))`\n",
    "\n",
    "tf.TensorShape (None) then the shape of the resulting tensor will be an unknown. Variable can be updated.\n",
    "\n",
    "`mammal = tf.Variable(\"Elephant\", dtype=tf.string)`\n",
    "\n",
    "`its_complicated = tf.Variable(4 + 3j, dtype=tf.complex64)`\n",
    "\n",
    "`first_primes = tf.Variable([2,3,4,5,6,7,11], dtype=tf.int32)`\n",
    "\n",
    "`linear_squares = tf.Variable([[4, 9], [16, 25]], dtype=tf.int32)`\n",
    "\n",
    "### Use tf.constant to create various kinds of tensors\n",
    "\n",
    ">Similarly, you can create tensors using tf.constant and notice that constant is spelt with a lowercase c whereas tf.Variable was spelled with a capital V. Also note that a key difference with tf constant is that you will not be able to modify the values once this tensor is created. \n",
    "\n",
    "`tensor = tf.constant([1, 2, 3])`\n",
    "\n",
    "`tensor = tf.constant([1, 2, 3, 4, 5, 6], shape=(2, 3))`\n",
    "\n",
    "`tensor = tf.constant(-1.0, shape=[2, 3])`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Math operations with tensors\n",
    "\n",
    "* tf.add\n",
    "* tf.square()\n",
    "* tf.reduce_sum()\n",
    "* tf.mul()\n",
    "* and more..\n",
    "\n",
    "**Operator overloading** is also supported."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### [The first lab of the course.](C2_W1_Lab_1_basic-tensors.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}