<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Jaehyek Choi</title>
    <description></description>
    <link>https://jaehyek.github.io/</link>
    <atom:link href="https://jaehyek.github.io/feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Wed, 12 Apr 2017 12:43:28 +0000</pubDate>
    <lastBuildDate>Wed, 12 Apr 2017 12:43:28 +0000</lastBuildDate>
    <generator>Jekyll v3.4.3</generator>
    
      <item>
        <title>Install Tensorflow and OpenCV on Raspberry PI 3</title>
        <description>다음의 내용은 Raspberry PI 3 에 Tensorflow을 설치하는 순서를 알려준다.

&lt;ol&gt;
  &lt;li&gt;
    &lt;a href=&quot;https://ubuntu-mate.org/raspberry-pi/&quot;&gt;https://ubuntu-mate.org/raspberry-pi/&lt;/a&gt; 에서  Ubuntu Mate을 설치한다. &lt;br /&gt;
 기본적으로 Python 2.7, 3.5 가 설치되어 있다. &lt;br /&gt;
 arm7l 용으로 build 되어 있는 tensorflow는 &lt;a href=&quot;https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/&quot;&gt;https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/&lt;/a&gt; &lt;br /&gt;
 에서 구할 수 있는 데 Python 3.4이다.  &lt;br /&gt;
 ( Python 3.5용은 별도로 compile해야 하므로 시간이 많이 걸린다. 거의 할 수 없다. ) &lt;br /&gt;
 해서 miniconda을 설치하면서 Python 3.4을 설치하고 tensorflow for Python 3.4을 설치한다. &lt;br /&gt;
  &lt;/li&gt;
  &lt;li&gt;miniconda을 설치한다.  &lt;br /&gt;
 &lt;a href=&quot;https://repo.continuum.io/miniconda/&quot;&gt;https://repo.continuum.io/miniconda/&lt;/a&gt; 에서 Miniconda3-3.16.0-Linux-armv7l.sh 을 다운 받는다. &lt;br /&gt;
 &lt;a href=&quot;https://www.continuum.io/blog/developer/anaconda-raspberry-pi&quot;&gt;https://www.continuum.io/blog/developer/anaconda-raspberry-pi&lt;/a&gt; 을 참조하면,  &lt;br /&gt;
    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ wget http://repo.continuum.io/miniconda/Miniconda3-3.16.0-Linux-armv7l.sh &amp;lt;br/&amp;gt;
$ md5sum Miniconda3-3.16.0-Linux-armv7l.sh &amp;lt;br/&amp;gt;
$ /bin/bash Miniconda3-3.16.0-Linux-armv7l.sh
&lt;/code&gt;&lt;/pre&gt;
    &lt;/div&gt;
  &lt;/li&gt;
  &lt;li&gt;path에 conda/bin을 추가한다.   &lt;br /&gt;
    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;export PATH=&quot;/home/jaehyek/miniconda3/bin:$PATH&quot;
&lt;/code&gt;&lt;/pre&gt;
    &lt;/div&gt;
  &lt;/li&gt;
  &lt;li&gt;
    Tensorflow 을 설치한다. &lt;br /&gt;
 &lt;a href=&quot;https://github.com/samjabrahams/tensorflow-on-raspberry-pi&quot;&gt;https://github.com/samjabrahams/tensorflow-on-raspberry-pi&lt;/a&gt; 을 참조하여 설치한다.

    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; $ sudo apt-get update
 $ sudo apt-get install python3-pip python3-dev
    
 $ wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.0.1/tensorflow-1.0.1-cp34-cp34m-linux_armv7l.whl &amp;lt;br/&amp;gt;
 $ pip3 install --user tensorflow-1.0.1-cp34-cp34m-linux_armv7l.whl &amp;lt;br/&amp;gt;
    
&lt;/code&gt;&lt;/pre&gt;
    &lt;/div&gt;
  &lt;/li&gt;
  &lt;li&gt;
    OpenCV을 설치한다.  &lt;br /&gt;
&lt;a href=&quot;http://www.emindlab.com/raspberry-pi/opencv-3-1-0-raspberry-pi.html&quot;&gt;http://www.emindlab.com/raspberry-pi/opencv-3-1-0-raspberry-pi.html&lt;/a&gt; 여기를 참조한다.&lt;br /&gt;

    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; sudo apt-get update
 sudo apt-get upgrade
    
 sudo apt-get install guvcview synaptic python-dev python-numpy python-scipy python-matplotlib 
 sudo apt-get install python-pandas python-nose build-essential cmake pkg-config
 sudo apt-get install default-jdk ant libgtkglext1-dev bison libqt4-dev-tools libqt4-dev libqt4-core libqt4-gui v4l-utils
 sudo apt-get install qtcreator
    
 sudo wget http://liquidtelecom.dl.sourceforge.net/project/opencvlibrary/opencv-unix/3.1.0/opencv-3.1.0.zip
 sudo unzip opencv-3.1.0.zip
 cd opencv-3.1.0
 sudo mkdir build
 cd build
    
 sudo cmake -D CMAKE_BUILD_TYPE=RELEASE -D INSTALL_C_EXAMPLES=ON –D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON 
  -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_OPENGL=ON -D WITH_V4L=ON –D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_TBB=ON ..
 sudo make
 sudo make install
 sudo nano /etc/ld.so.conf.d/opencv.conf
 /usr/local/lib
    
 Press Control + X to save
    
 sudo ldconfig
    
 sudo nano /etc/bash.bashrc
    
 export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig
    
    
 cd opencv-3.1.0/samples/cpp
    
 g++ -o facedetect facedetect.cpp `pkg-config opencv --cflags --libs`
    
 ./facedetect
    
 https://docs.google.com/document/d/1OcVoQi8UJ2bCtJ2nFPkI-eZi9kuQoqthpOwpCpLGwJE/edit?usp=sharing
    
&lt;/code&gt;&lt;/pre&gt;
    &lt;/div&gt;
  &lt;/li&gt;
  &lt;li&gt;
    Camera을 enable한다. 그리고 확인해 본다  &lt;br /&gt;
&lt;a href=&quot;https://larrylisky.com/2016/11/24/enabling-raspberry-pi-camera-v2-under-ubuntu-mate/&quot;&gt;https://larrylisky.com/2016/11/24/enabling-raspberry-pi-camera-v2-under-ubuntu-mate/&lt;/a&gt; 여기를 참조한다.  &lt;br /&gt;

    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt; sudo apt-get update
 sudo apt-get upgrade
 sudo apt-get install raspi-config rpi-update
 sudo raspi-config
 make sure /boot/config.txt
      start_x=1
      gpu_mem=128
    
 만일 다음과 같이 error가 난다면, 
      mmal: mmal_component_create_core: could not find component 'vc.camera_info'
      mmal: Failed to create camera_info component
      mmal: mmal_component_create_core: could not find component 'vc.ril.camera'
      mmal: Failed to create camera component
      mmal: main: Failed to create camera component
      mmal: Failed to run camera app. Please check for firmware updates
 old version을 다운한다.
 sudo rpi-update 667cfabe63bc663383559ef88317e86f9bd41e45
    
 그리고 다음을 수행한다.
      git clone https://github.com/raspberrypi/userland.git
      cd userland
      ./buildme
      touch ~/.bash_aliases
      echo -e 'PATH=$PATH:/opt/vc/bin\nexport PATH' &amp;gt;&amp;gt; ~/.bash_aliases
      echo -e 'LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/vc/lib\nexport LD_LIBRARY_PATH' &amp;gt;&amp;gt; ~/.bash_aliases
      source ~/.bashrc
      sudo ldconfig
      sudo reboot now
    
 마지막 확인하기
 $ raspivid -p 0,0,640,480 -t 0
    
 python에서 opencv을 열어서 확인하기.
    
 webcam.py
    
 import cv2
    
 def show_webcam(mirror=False):
     cam = cv2.VideoCapture(0)
     while True:
         ret_val, img = cam.read()
         if mirror: 
             img = cv2.flip(img, 1)
         cv2.imshow('my webcam', img)
         if cv2.waitKey(1) == 27: 
             break  # esc to quit
     cv2.destroyAllWindows()
    
 def main():
     show_webcam(mirror=True)
    
 if __name__ == '__main__':
     main()
    
&lt;/code&gt;&lt;/pre&gt;
    &lt;/div&gt;
  &lt;/li&gt;
&lt;/ol&gt;
</description>
        <pubDate>Sat, 01 Apr 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/04/01/Deep-Learning-Install-Tensorflow-OpenCV-RaspberryPi3/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/04/01/Deep-Learning-Install-Tensorflow-OpenCV-RaspberryPi3/</guid>
        
        <category>Deep-Learning</category>
        
        <category>Tensorflow</category>
        
        <category>OpenCV</category>
        
        <category>Raspberry</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>Install Tensorflow on Windows</title>
        <description>다음의 내용은 &lt;a href=&quot;http://tmmse.xyz/2017/03/01/tensorflow-keras-installation-windows-linux-macos/&quot;&gt;여기&lt;/a&gt;을 참조하자.

&lt;ol&gt;
  &lt;li&gt;Anaconda 설치&lt;/li&gt;
  &lt;li&gt;CUDA설치 ( https://developer.nvidia.com/cuda-downloads )&lt;/li&gt;
  &lt;li&gt;cuDNN 설치 ver 8.0
 경로 추가 : c:\Program Files\NVIDiA GPU Computing Toolkit\CUDA\v8.0&lt;/li&gt;
  &lt;li&gt;관리자 권한에서 다음 실행.
  conda create -n tensorflow_in_cpu python=3.5
 혹은
  conda create -n tensorflow_in_gpu python=3.5&lt;/li&gt;
  &lt;li&gt;conda 환경 만든 곳에 tensorflow 설치하기
  activate tensorflow_in_cpu
  pip install tensorflow  ( 오류시 pip install tensorflow=0.12.1)
  혹은 
  activate tensorflow_in_gpu
  pip install tensorflow-gpu 
  ** pip3으로도 설치가능.
    &lt;h1 id=&quot;여기서부터-tensorflow_in_cpu-환경-설치-순서-중요&quot;&gt;여기서부터 (tensorflow_in_cpu) 환경. 설치 순서 중요&lt;/h1&gt;
  &lt;/li&gt;
  &lt;li&gt;Pycharm에서 tensorflow project만들기
  c:\Anaconda3\envs\tensorflow(_in_cpu)\python.exe을 선택&lt;/li&gt;
  &lt;li&gt;tensorflow 버전을 업데이트 시키고 싶다면
    &lt;ul&gt;
      &lt;li&gt;activate로 conda환경 들어 가신 후&lt;/li&gt;
      &lt;li&gt;pip install –upgrade tensorflow 혹은 
 pip install –upgrade tensorflow-gpu&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

conda install pandas matplotlib scikit-learn&lt;br /&gt;
pip install keras&lt;br /&gt;
conda install jupyter notebook
</description>
        <pubDate>Tue, 28 Feb 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/02/28/Deep-Learning-Install-Tensorflow-Windows/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/02/28/Deep-Learning-Install-Tensorflow-Windows/</guid>
        
        <category>Deep-Learning</category>
        
        <category>Tensorflow</category>
        
        <category>Windows</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>Tensorflow 0.x =&gt; 1.0 Migration Guide</title>
        <description>다음의 내용은 &lt;a href=&quot;https://github.com/bluedisk/tensorflow-study&quot;&gt;여기&lt;/a&gt;에서 퍼 왔습니다.

1.0.0rc0 정식 출시에따라 겪게되는 맨붕 사태를 정리해 봤습니다.

&lt;h2 id=&quot;common&quot;&gt;Common&lt;/h2&gt;

&lt;h3 id=&quot;초기화--initialize_all_variables--global_variables_initializer-or-local_variables_initializer&quot;&gt;초기화 : initialize_all_variables =&amp;gt; global_variables_initializer or local_variables_initializer&lt;/h3&gt;

2017-3-2 삭제 예정. 아직은 경고만 뜹니다만 미리 변경을 추천드립니다.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;saver&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;train&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Saver&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;all_variables&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;saver&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;train&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Saver&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;global_variables&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h2 id=&quot;saver&quot;&gt;Saver&lt;/h2&gt;

&lt;h3 id=&quot;all_variables--global_variables&quot;&gt;all_variables =&amp;gt; global_variables&lt;/h3&gt;

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;sess&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;run&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;initialize_all_variables&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;sess&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;run&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;global_variables_initializer&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
또는
&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;sess&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;run&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;local_variables_initializer&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h3 id=&quot;write_version-tftrainsaverdefv1--tftrainsaverdefv2&quot;&gt;write_version tf.train.SaverDef.V1 =&amp;gt; tf.train.SaverDef.V2&lt;/h3&gt;

저장 포멧 변경으로 하위 호환이 사라짐. 하위 호환이 필요없다면 무시 가능

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;train&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Saver&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;max_to_keep&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;호환이 필요하다면 다음과 같이 해야함(미지정 시 기본 포멧이 V2로 저장됨)&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;train&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Saver&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;max_to_keep&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;200&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;write_version&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;train&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;SaverDef&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;V1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; 
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;

전체적으로 summery 관련 함수들이 tf에서 tf.summery로 이동되고 Summery prefix가 삭제되는 형태의 단순 이름 변경.

&lt;h3 id=&quot;summarywriter--filewriter&quot;&gt;SummaryWriter =&amp;gt; FileWriter&lt;/h3&gt;

단순히 이름만 변경됨.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;train_writer&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;train&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;SummaryWriter&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'log/'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sess&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;graph&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;train_writer&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;summary&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;FileWriter&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'log/'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;sess&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;graph&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h3 id=&quot;merge_all_summaries--summarymerge_all&quot;&gt;merge_all_summaries =&amp;gt; summary.merge_all&lt;/h3&gt;

단순히 이름만 변경됨.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;merged&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;merge_all_summaries&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;merged&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;summary&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;merge_all&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h3 id=&quot;scalar_summary--summaryscalar&quot;&gt;scalar_summary =&amp;gt; summary.scalar&lt;/h3&gt;

단순히 이름만 변경됨. 비슷한 류의 summary 함수들 모두 동일하게 
&lt;code class=&quot;highlighter-rouge&quot;&gt;*_summary =&amp;gt; summary.*&lt;/code&gt;

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;cost_summary&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;scalar_summary&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'Cost'&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;cost&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;cost_summary&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;summary&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;scalar&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;'Cost, cost)&lt;/span&gt;&lt;span class=&quot;err&quot;&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h2 id=&quot;optimizer&quot;&gt;Optimizer&lt;/h2&gt;

&lt;h3 id=&quot;softmax_cross_entropy_with_logits&quot;&gt;softmax_cross_entropy_with_logits&lt;/h3&gt;

Argument가 암시적 지정 방식을 더이상 지원하지 않습니다. 명시적 지정이 필요합니다.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;cost&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;softmax_cross_entropy_with_logits&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;pred&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;y&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;cost&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;softmax_cross_entropy_with_logits&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;logits&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;pred&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;labels&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;y&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h2 id=&quot;rnn과-rnn에서-많이-쓰는-함수&quot;&gt;RNN과 RNN에서 많이 쓰는 함수&lt;/h2&gt;

&lt;h3 id=&quot;tfnnrnn_cell---tfcontribrnn&quot;&gt;tf.nn.rnn_cell.*  =&amp;gt; tf.contrib.rnn.*&lt;/h3&gt;

패키지명 단순 이동.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;encoDecoCell&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;rnn_cell&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;BasicLSTMCell&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;args&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;hiddenSize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;state_is_tuple&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;encoDecoCell&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;contrib&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;rnn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;BasicLSTMCell&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;args&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;hiddenSize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;state_is_tuple&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;True&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h3 id=&quot;tfnnseq2seq--tfcontriblegacy_seq2seq&quot;&gt;tf.nn.seq2seq.* =&amp;gt; tf.contrib.legacy_seq2seq.*&lt;/h3&gt;

패키지명 단순 이동. legacy가 붙은걸 봐선 권장하는 방식이 아닐진데… 
더 이쁘게 업그레이드 하는 방법 아시는분 추천 부탁드립니다.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;decoderOutputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;states&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;seq2seq&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embedding_rnn_seq2seq&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;encoderInputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; 
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;decoderInputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; 
     &lt;span class=&quot;n&quot;&gt;encoDecoCell&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;textData&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;getVocabularySize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(),&lt;/span&gt;
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;textData&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;getVocabularySize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(),&lt;/span&gt;  
     &lt;span class=&quot;n&quot;&gt;embedding_size&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;args&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embeddingSize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;  
     &lt;span class=&quot;n&quot;&gt;output_projection&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outputProjection&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;getWeights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;outputProjection&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;None&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
     &lt;span class=&quot;n&quot;&gt;feed_previous&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;bool&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;args&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;test&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;  
 &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;decoderOutputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;states&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;contrib&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;legacy_seq2seq&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embedding_rnn_seq2seq&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;encoderInputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; 
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;decoderInputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; 
     &lt;span class=&quot;n&quot;&gt;encoDecoCell&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;textData&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;getVocabularySize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(),&lt;/span&gt;
     &lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;textData&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;getVocabularySize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(),&lt;/span&gt;  
     &lt;span class=&quot;n&quot;&gt;embedding_size&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;args&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embeddingSize&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;  
     &lt;span class=&quot;n&quot;&gt;output_projection&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outputProjection&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;getWeights&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;outputProjection&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt; &lt;span class=&quot;bp&quot;&gt;None&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;
     &lt;span class=&quot;n&quot;&gt;feed_previous&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;bool&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;bp&quot;&gt;self&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;args&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;test&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;  
 &lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h3 id=&quot;tfsplitsplit_dim-num_splits-value-tfsplitvalue-num_or_size_splits-axis---순서-바뀜-&quot;&gt;tf.split(split_dim, num_splits, value) –&amp;gt; tf.split(value, num_or_size_splits, axis)  !!!! 순서 바뀜 !!!!&lt;/h3&gt;

잘돌아가던 rnn이 업그레이드 후 이상동작을 보인다면 95.135%는 여기가 문제입니다.

&lt;strong&gt;v0.x 일반적인 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;inputs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;split&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;seq_length&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embedding_lookup&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embedding&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;input_data&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;# inputs = tf.split(1, seq_length, tf.nn.embedding_lookup(embedding, input_data))&lt;/span&gt;
&lt;span class=&quot;n&quot;&gt;inputs&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;split&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;nn&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embedding_lookup&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;embedding&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;input_data&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;seq_length&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h3 id=&quot;tfconcatconcat_dim-values--tfconcat_v2values-axis&quot;&gt;tf.concat(concat_dim, values) –&amp;gt; tf.concat_v2(values, axis)&lt;/h3&gt;

다행히 v2 함수가 따로 준비되어서 당분간 기존 함수로도 수정 없이 무난히 동작합니다.

&lt;strong&gt;v0.x에 잘돌아가던 코드&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;output&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reshape&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;concat&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;outputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;rnn_size&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;strong&gt;v1.0.0rc0&lt;/strong&gt;

&lt;div class=&quot;language-python highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;output&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;reshape&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;tf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;concat_v2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;outputs&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;),&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;-&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;rnn_size&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;hr /&gt;

&lt;h2 id=&quot;tensorflowbloc-공식-발표에-나왔으나-아직-미정리-ㅠㅜ&quot;&gt;tensorflow.bloc 공식 발표에 나왔으나 아직 미정리 ㅠㅜ&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;tf.pack / tf.unpack –&amp;gt; tf.stack / tf.unstack
    &lt;ul&gt;
      &lt;li&gt;v0.12: stack/unstack 추가, pack/unpack 은 stack/unstack 을 단순 wrapping&lt;/li&gt;
      &lt;li&gt;master: pack/unpack 에 deprecated 메세지 추가&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.concat(concat_dim, values) –&amp;gt; tf.concat_v2(values, axis)
    &lt;ul&gt;
      &lt;li&gt;v0.12: concat_v2 추가&lt;/li&gt;
      &lt;li&gt;master: concat 에 deprecated 메세지 추가&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.sparse_split(split_dim, num_split, sp_input) –&amp;gt; tf.sparse_split(sp_input, num_split, axis)
    &lt;ul&gt;
      &lt;li&gt;master: sparse_split(sp_input, num_split, axis) 로 변경&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.reverse(tensor, dims) –&amp;gt; tf.reverse(tensor, axis)
    &lt;ul&gt;
      &lt;li&gt;v0.12: reverse_v2(tensor, axis) 추가&lt;/li&gt;
      &lt;li&gt;master: reverse 가 바뀌고, v1.0 이후 reverse_v2 가 deprecated 될 예정&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.round –&amp;gt; banker’s rounding
    &lt;ul&gt;
      &lt;li&gt;v0.12: 파이썬의 banker’s rounding 으로 변경, 짝수로 반올림/내림&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;dimension, dim, ~indices, ~dim, ~axes 키워드 파라미터 –&amp;gt; axis 로 통일
    &lt;ul&gt;
      &lt;li&gt;v0.12: 기존 파라미터와 axis 모두 유지, 향후 기존 파라미터 삭제 예정&lt;/li&gt;
      &lt;li&gt;tf.argmax: dimension –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.argmin: dimension –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.count_nonzero: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.expand_dims: dim –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_all: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_any: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_join: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_logsumexp: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_max: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_mean: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_min: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_prod: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reduce_sum: reduction_indices –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.reverse_sequence: batch_dim –&amp;gt; batch_axis, seq_dim –&amp;gt; seq_axis&lt;/li&gt;
      &lt;li&gt;tf.sparse_concat: concat_dim –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.sparse_reduce_sum: reduction_axes –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.sparse_reduce_sum_sparse: reduction_axes –&amp;gt; axis&lt;/li&gt;
      &lt;li&gt;tf.sparse_split: split_dim –&amp;gt; axis&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.listdiff –&amp;gt; tf.setdiff1d
    &lt;ul&gt;
      &lt;li&gt;v0.12: setdiff1d 추가&lt;/li&gt;
      &lt;li&gt;master: listdiff 에 deprecated 메세지 추가&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.select –&amp;gt; tf.where
    &lt;ul&gt;
      &lt;li&gt;v0.12: where 추가&lt;/li&gt;
      &lt;li&gt;master: select 삭제&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.inv –&amp;gt; tf.reciprocal
    &lt;ul&gt;
      &lt;li&gt;v0.12: inv –&amp;gt; reciprocal 이름 변경&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.SparseTensor.shape –&amp;gt; tf.SparseTensor.dense_shape
    &lt;ul&gt;
      &lt;li&gt;master: shape –&amp;gt; dense_shape 이름변경&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;tf.SparseTensorValue.shape –&amp;gt; tf.SparseTensorValue.dense_shape
    &lt;ul&gt;
      &lt;li&gt;master: shape –&amp;gt; dense_shape 이름 변경&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;zeros_initializer –&amp;gt; 함수 리턴으로 변경
    &lt;ul&gt;
      &lt;li&gt;master: ones_initializer 와 동일한 리턴 형식으로 변경&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;v0.12: TensorFlowEstimator() 가 삭제되었습니다. 대신 Estimator() 가 권장됩니다.&lt;/li&gt;
&lt;/ul&gt;
</description>
        <pubDate>Mon, 30 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/30/Deep-Learning-Tensorflow-Migration/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/30/Deep-Learning-Tensorflow-Migration/</guid>
        
        <category>Deep-Learning</category>
        
        <category>Tensorflow</category>
        
        <category>Migration</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>eMMC UFS Issues 17</title>
        <description>&lt;h2 id=&quot;component-feature&quot;&gt;Component Feature&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;&lt;/li&gt;
  &lt;li&gt;Controller :&lt;/li&gt;
  &lt;li&gt;Hy&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;failure-symptom&quot;&gt;Failure Symptom&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;No power-on on smart phone.&lt;/li&gt;
  &lt;li&gt;infinite loop booting&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;possible-error-type&quot;&gt;Possible Error Type&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;NAND Level : bit flip error due to particle or fab error&lt;/li&gt;
  &lt;li&gt;FW Level : bit flip from FW error&lt;/li&gt;
  &lt;li&gt;emmc internal SRAM Level : bit flip from SRAM unstability&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;conclusion-after-inspection-of-bit-flip&quot;&gt;Conclusion after inspection of bit flip&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;The NAND level and FW Level have on doubt.&lt;/li&gt;
  &lt;li&gt;The cause suspicion is that the bit flip was casused from SRAM data failure , and the data was written to NAND , the wrong data was transferd at booting time.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;what-is-the-soft-error-&quot;&gt;What is the soft error ?&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;is bit flip&lt;/li&gt;
  &lt;li&gt;but  if rewrite , the error can be recovered.&lt;/li&gt;
&lt;/ul&gt;

&lt;table&gt;
  &lt;thead&gt;
    &lt;tr&gt;
      &lt;th&gt;Cause&lt;/th&gt;
      &lt;th&gt;Description&lt;/th&gt;
    &lt;/tr&gt;
  &lt;/thead&gt;
  &lt;tbody&gt;
    &lt;tr&gt;
      &lt;td&gt;A. Alpha particles from package&lt;/td&gt;
      &lt;td&gt;caused by radioactive materials Trace within package&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;B. Cosmic rays creating energetic neutrons and protons&lt;/td&gt;
      &lt;td&gt;caused by high energy from Atmospheric environment&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;C. Thermal&lt;/td&gt;
      &lt;td&gt;The energy stored in the cell is lost over time, as the temperature accelerates and the original data is transformed&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt;D. Random Noise or Signal Integrity(Include Current Spike)&lt;/td&gt;
      &lt;td&gt;Random noise and signal integration problem wih power and signal&lt;/td&gt;
    &lt;/tr&gt;
    &lt;tr&gt;
      &lt;td&gt; &lt;/td&gt;
      &lt;td&gt;can occurs when the voltage is dropped to under operation voltage Momentarily&lt;/td&gt;
    &lt;/tr&gt;
  &lt;/tbody&gt;
&lt;/table&gt;

&lt;img src=&quot;/img/2017-01-20-eMMC-UFS-Issues-17/001.JPG&quot; width=&quot;300&quot; /&gt;&lt;img src=&quot;/img/2017-01-20-eMMC-UFS-Issues-17/002.JPG&quot; width=&quot;300&quot; /&gt;&lt;img src=&quot;/img/2017-01-20-eMMC-UFS-Issues-17/003.JPG&quot; width=&quot;300&quot; /&gt;&lt;img src=&quot;/img/2017-01-20-eMMC-UFS-Issues-17/004.JPG&quot; width=&quot;300&quot; /&gt;

&lt;h2 id=&quot;instant-voltage-drop-on-emmc&quot;&gt;Instant Voltage Drop on eMMC&lt;/h2&gt;
&lt;ul&gt;
  &lt;li&gt;A potent suspect factor&lt;/li&gt;
  &lt;li&gt;the evidence&lt;/li&gt;
&lt;/ul&gt;

&lt;img src=&quot;/img/2017-01-20-eMMC-UFS-Issues-17/005.JPG&quot; width=&quot;400&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;VCCQ ( 1.2V ) voltage was dropped momentarily between 30nS and 50nS&lt;/li&gt;
&lt;/ul&gt;

&lt;h2 id=&quot;how-to-prevent-the-bit-flip-on-sram-of-emmc&quot;&gt;How to prevent the bit flip on SRAM of eMMC&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;install a voltage regulator in front of VCCQ&lt;/li&gt;
  &lt;li&gt;detect the instant voltage drop, and then reset system.&lt;/li&gt;
  &lt;li&gt;checksum algorithm applying to SRAM&lt;/li&gt;
&lt;/ul&gt;

1.In Normal Operation
&lt;ul&gt;
  &lt;li&gt;Critical management tables are being verified by checksum algorithm&lt;/li&gt;
  &lt;li&gt;Before any update or flush to NAND, checksum is calculated and verified&lt;/li&gt;
&lt;/ul&gt;

2.In Sleep Mode(cmd5 and AutoSleep )
&lt;ul&gt;
  &lt;li&gt;While entering and exiting sleep, RAM refresh algorithm is activated&lt;/li&gt;
  &lt;li&gt;Read Data –&amp;gt; Detect bit flip –&amp;gt; Recover if needed&lt;/li&gt;
&lt;/ul&gt;

&lt;img src=&quot;/img/2017-01-20-eMMC-UFS-Issues-17/006.JPG&quot; alt=&quot;006&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;Integrity of data is assured during data flow within the ASIC&lt;/li&gt;
  &lt;li&gt;No performace impact&lt;/li&gt;
&lt;/ul&gt;

</description>
        <pubDate>Fri, 20 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/20/eMMC-UFS-Issues-17/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/20/eMMC-UFS-Issues-17/</guid>
        
        <category>eMMC</category>
        
        <category>UFS</category>
        
        <category>Issues</category>
        
        
        <category>eMMC-UFS</category>
        
      </item>
    
      <item>
        <title>Deep Learning summary from http://hunkim.github.io/ml/ (4)</title>
        <description>&lt;h1 id=&quot;recurrent-neural-network&quot;&gt;Recurrent Neural Network&lt;/h1&gt;

&lt;h4 id=&quot;refer-to-httpcs231nstanfordedu&quot;&gt;refer to http://cs231n.stanford.edu/&lt;/h4&gt;

&lt;ol&gt;
  &lt;li&gt;
    We can process a sequence of vectors x by applying a recurrence formula at every time step:
  &lt;/li&gt;
  &lt;li&gt;
    Notice: the same function and the same set of parameters are used at every time step.
  &lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-04/001.JPG&quot; alt=&quot;001&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-04/002.JPG&quot; alt=&quot;002&quot; /&gt;

</description>
        <pubDate>Sun, 15 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-04/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-04/</guid>
        
        <category>Deep-Learning</category>
        
        <category>summary</category>
        
        <category>hunkim</category>
        
        <category>github</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>Deep Learning summary from http://hunkim.github.io/ml/ (3)</title>
        <description>&lt;h1 id=&quot;convolutional-neural-networks&quot;&gt;Convolutional Neural Networks&lt;/h1&gt;

&lt;h4 id=&quot;refer-to-httpcs231nstanfordedu&quot;&gt;refer to http://cs231n.stanford.edu/&lt;/h4&gt;

&lt;ol&gt;
  &lt;li&gt;Let’s focus on a small area only (5x5x3)&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/001.JPG&quot; alt=&quot;001&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;Get one number using the filter&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/002.JPG&quot; alt=&quot;002&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;Let’s look at other areas with the same filter (w)&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/003.JPG&quot; alt=&quot;003&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;In practice: Common to zero pad the border&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/004.JPG&quot; alt=&quot;004&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;Swiping the entire image&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/005.JPG&quot; alt=&quot;005&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;Convolution layers&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/006.JPG&quot; alt=&quot;006&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/007.JPG&quot; alt=&quot;007&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;Pooling layer (sampling) &amp;amp; MAX POOLING&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/008.JPG&quot; alt=&quot;008&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/009.JPG&quot; alt=&quot;009&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;Fully Connected Layer (FC layer)
Contains neurons that connect to the entire input volume, as in ordinary Neural Networks&lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/010.JPG&quot; alt=&quot;010&quot; /&gt;

&lt;ol&gt;
  &lt;li&gt;ConvNetJS demo: training on CIFAR-10]&lt;/li&gt;
&lt;/ol&gt;

refer to &lt;a href=&quot;http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html&quot;&gt;http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html&lt;/a&gt;

&lt;h1 id=&quot;cnn-case-study&quot;&gt;CNN Case Study&lt;/h1&gt;

&lt;h4 id=&quot;case-study-lenet-5-lecun-et-al-1998&quot;&gt;Case Study: LeNet-5 [LeCun et al., 1998]&lt;/h4&gt;

&lt;h4 id=&quot;case-study-alexnet-krizhevsky-et-al-2012&quot;&gt;Case Study: AlexNet [Krizhevsky et al. 2012]&lt;/h4&gt;

&lt;h4 id=&quot;case-study-googlenet-szegedy-et-al-2014&quot;&gt;Case Study: GoogLeNet [Szegedy et al., 2014]&lt;/h4&gt;

&lt;h4 id=&quot;case-study-resnet-he-et-al-2015&quot;&gt;Case Study: ResNet [He et al., 2015]&lt;/h4&gt;

&lt;ul&gt;
  &lt;li&gt;Slide from Kaiming He’s recent presentation https://www.youtube.com/watch?v=1PGLj-uKT1w&lt;/li&gt;
  &lt;li&gt;ILSVRC 2015 winner (3.6% top 5 error)&lt;/li&gt;
  &lt;li&gt;2-3 weeks of training on 8 GPU machine&lt;/li&gt;
  &lt;li&gt;at runtime: faster than a VGGNet! (even though it has 8x more layers)&lt;/li&gt;
&lt;/ul&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/011.JPG&quot; alt=&quot;011&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/012.JPG&quot; alt=&quot;012&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/013.JPG&quot; alt=&quot;013&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-03/014.JPG&quot; alt=&quot;014&quot; /&gt;

</description>
        <pubDate>Sun, 15 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-03/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-03/</guid>
        
        <category>Deep-Learning</category>
        
        <category>summary</category>
        
        <category>hunkim</category>
        
        <category>github</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>Deep Learning summary from http://hunkim.github.io/ml/ (2)</title>
        <description>&lt;h4 id=&quot;canadian-institute-for-advanced-research-cifar&quot;&gt;Canadian Institute for Advanced Research (CIFAR)&lt;/h4&gt;

&lt;h4 id=&quot;google-deepminds-deep-q-learning-playing-atari-breakout&quot;&gt;Google DeepMind’s Deep Q-learning playing Atari Breakout&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/001.JPG&quot; alt=&quot;001&quot; /&gt;
&lt;a href=&quot;https://youtu.be/V1eYniJ0Rnk&quot;&gt;https://youtu.be/V1eYniJ0Rnk&lt;/a&gt;

&lt;h4 id=&quot;multinomial-classification&quot;&gt;Multinomial classification&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/002.JPG&quot; alt=&quot;002&quot; /&gt;

&lt;h4 id=&quot;back-propagation&quot;&gt;Back propagation&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/003.JPG&quot; alt=&quot;003&quot; /&gt;

&lt;h3 id=&quot;5-steps-of-using-tensorboard&quot;&gt;5 steps of using tensorboard&lt;/h3&gt;
&lt;ol&gt;
  &lt;li&gt;From TF graph, decide which node you want to annotate
    &lt;ul&gt;
      &lt;li&gt;with tf.name_scope(“test”) as scope:&lt;/li&gt;
      &lt;li&gt;tf.histogram_summary(“weights”, W), tf.scalar_summary(“accuracy”, accuracy)&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Merge all summaries
    &lt;ul&gt;
      &lt;li&gt;merged = tf.merge_all_summaries()&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Create writer
    &lt;ul&gt;
      &lt;li&gt;writer = tf.train.SummaryWriter(“/tmp/mnist_logs”, sess.graph_def)&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Run summary merge and add_summary
    &lt;ul&gt;
      &lt;li&gt;summary = sess.run(merged, …); writer.add_summary(summary);&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Launch Tensorboard
    &lt;ul&gt;
      &lt;li&gt;tensorboard –logdir=/tmp/mnist_logs&lt;/li&gt;
      &lt;li&gt;You can navigate to http://0.0.0.0:6006&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/004.JPG&quot; alt=&quot;004&quot; /&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;import tensorflow as tf
import numpy as np

xy = np.loadtxt('07train.txt', unpack=True)
x_data = np.transpose(xy[0:-1])
y_data = np.reshape(xy[-1], (4, 1))

print x_data
print y_data

X = tf.placeholder(tf.float32, name='x-input')
Y = tf.placeholder(tf.float32, name='y-input')

w1 = tf.Variable(tf.random_uniform([2, 5], -1.0, 1.0), name='weight1')
w2 = tf.Variable(tf.random_uniform([5, 10], -1.0, 1.0), name='weight2')
w3 = tf.Variable(tf.random_uniform([10, 10], -1.0, 1.0), name='weight3')
w4 = tf.Variable(tf.random_uniform([10, 10], -1.0, 1.0), name='weight4')
w5 = tf.Variable(tf.random_uniform([10, 10], -1.0, 1.0), name='weight5')
w6 = tf.Variable(tf.random_uniform([10, 10], -1.0, 1.0), name='weight6')
w7 = tf.Variable(tf.random_uniform([10, 10], -1.0, 1.0), name='weight7')
w8 = tf.Variable(tf.random_uniform([10, 1], -1.0, 1.0), name='weight8')

b1 = tf.Variable(tf.zeros([5]), name=&quot;Bias1&quot;)
b3 = tf.Variable(tf.zeros([10]), name=&quot;Bias3&quot;)
b2 = tf.Variable(tf.zeros([10]), name=&quot;Bias2&quot;)
b4 = tf.Variable(tf.zeros([10]), name=&quot;Bias4&quot;)
b5 = tf.Variable(tf.zeros([10]), name=&quot;Bias5&quot;)
b6 = tf.Variable(tf.zeros([10]), name=&quot;Bias6&quot;)
b7 = tf.Variable(tf.zeros([10]), name=&quot;Bias7&quot;)
b8 = tf.Variable(tf.zeros([1]), name=&quot;Bias8&quot;)

L2 = tf.nn.relu(tf.matmul(X, w1) + b1)
L3 = tf.nn.relu(tf.matmul(L2, w2) + b2)
L4 = tf.nn.relu(tf.matmul(L3, w3) + b3)
L5 = tf.nn.relu(tf.matmul(L4, w4) + b4)
L6 = tf.nn.relu(tf.matmul(L5, w5) + b5)
L7 = tf.nn.relu(tf.matmul(L6, w6) + b6)
L8 = tf.nn.relu(tf.matmul(L7, w7) + b7)

hypothesis = tf.sigmoid(tf.matmul(L8, w8) + b8)

with tf.name_scope('cost') as scope:
    cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1-Y) * tf.log(1 - hypothesis))
    cost_summ = tf.summary.scalar(&quot;cost&quot;, cost)

with tf.name_scope('train') as scope:
    a = tf.Variable(0.1)
    optimizer = tf.train.GradientDescentOptimizer(a)
    train = optimizer.minimize(cost)

w1_hist = tf.summary.histogram(&quot;weights1&quot;, w1)
w2_hist = tf.summary.histogram(&quot;weights2&quot;, w2)

b1_hist = tf.summary.histogram(&quot;biases1&quot;, b1)
b2_hist = tf.summary.histogram(&quot;biases2&quot;, b2)

y_hist = tf.summary.histogram(&quot;y&quot;, Y)

init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)

    merged = tf.summary.merge_all()
    writer = tf.summary.FileWriter(&quot;./logs/xor_logs&quot;, sess.graph)

    for step in xrange(20000):
        sess.run(train, feed_dict={X: x_data, Y: y_data})
        if step % 200 == 0:
            summary = sess.run(merged, feed_dict={X: x_data, Y: y_data})
            writer.add_summary(summary, step)
            print step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(w1), sess.run(w2)

    correct_prediction = tf.equal(tf.floor(hypothesis+0.5), Y)

    accuracy = tf.reduce_mean(tf.cast(correct_prediction, &quot;float&quot;))
    print sess.run([hypothesis, tf.floor(hypothesis+0.5), correct_prediction], feed_dict={X: x_data, Y: y_data})
    print &quot;accuracy&quot;, accuracy.eval({X: x_data, Y: y_data})
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h4 id=&quot;relu-rectified-linear-unit&quot;&gt;ReLU: Rectified Linear Unit&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;L1 = tf.sigmoid(tf.matmul(X, W1) + b1)&lt;/li&gt;
  &lt;li&gt;
    L1 = tf.nn.relu(tf.matmul(X, W1) + b1)
  &lt;/li&gt;
  &lt;li&gt;maxout = max(w1x + b1, w2x + b2 )&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;activation-functions-on-cifar-10&quot;&gt;Activation functions on CIFAR-10&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/005.JPG&quot; alt=&quot;005&quot; /&gt;

&lt;h4 id=&quot;need-to-set-the-initial-weight-values-wisely&quot;&gt;Need to set the initial weight values wisely&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;Not all 0’s&lt;/li&gt;
  &lt;li&gt;Challenging issue&lt;/li&gt;
  &lt;li&gt;Hinton et al. (2006) “A Fast Learning Algorithm for Deep Belief Nets”&lt;/li&gt;
  &lt;li&gt;Restricted Boatman Machine (RBM) :  not used these day&lt;/li&gt;
  &lt;li&gt;recreate input ( encoder/decoder )&lt;/li&gt;
  &lt;li&gt;Deep Belief Net ( initialized by RBM )&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;good-news&quot;&gt;Good news&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;No need to use complicated RBM for weight initializations&lt;/li&gt;
  &lt;li&gt;Simple methods are OK
    &lt;ul&gt;
      &lt;li&gt;Xavier initialization&lt;/li&gt;
      &lt;li&gt;He’s initialization&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
  &lt;li&gt;Makes sure the weights are ‘just right’, not too small, not too big&lt;/li&gt;
  &lt;li&gt;Using number of input (fan_in) and output (fan_out)&lt;/li&gt;
&lt;/ul&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;# Xavier initialization
# Glorot et al.2010
W = np.random.randn(fan_in, fan_out)/np.sqrt(fan_in)

# He et al.2015
W = np.random.randn(fan_in, fan_out)/np.sqrt(fan_in/2)
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;ul&gt;
  &lt;li&gt;prettytensor implementation&lt;/li&gt;
&lt;/ul&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;def xavier_init(n_inputs, n_outputs, uniform=True):
  &quot;&quot;&quot;Set the parameter initialization using the method described.
  This method is designed to keep the scale of the gradients roughly the same
  in all layers.
  Xavier Glorot and Yoshua Bengio (2010):
           Understanding the difficulty of training deep feedforward neural
           networks. International conference on artificial intelligence and
           statistics.
  Args:
    n_inputs: The number of input nodes into each output.
    n_outputs: The number of output nodes for each input.
    uniform: If true use a uniform distribution, otherwise use a normal.
  Returns:
    An initializer.
  &quot;&quot;&quot;
  if uniform:
    # 6 was used in the paper.
    init_range = math.sqrt(6.0 / (n_inputs + n_outputs))
    return tf.random_uniform_initializer(-init_range, init_range)
  else:
    # 3 gives us approximately the same limits as above since this repicks
    # values greater than 2 standard deviations from the mean.
    stddev = math.sqrt(3.0 / (n_inputs + n_outputs))
    return tf.truncated_normal_initializer(stddev=stddev)
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;ul&gt;
  &lt;li&gt;Activation functions and initialization on CIFAR-10&lt;/li&gt;
&lt;/ul&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/006.JPG&quot; alt=&quot;006&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;Geoffrey Hinton’s summary of findings up to today
    &lt;ul&gt;
      &lt;li&gt;Our labeled datasets were thousands of times too small.&lt;/li&gt;
      &lt;li&gt;Our computers were millions of times too slow&lt;/li&gt;
      &lt;li&gt;We initialized the weights in a stupid way&lt;/li&gt;
      &lt;li&gt;We used the wrong type of non-linearity&lt;/li&gt;
    &lt;/ul&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;overfitting&quot;&gt;Overfitting&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/007.JPG&quot; alt=&quot;007&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;Solutions for overfitting
    &lt;ul&gt;
      &lt;li&gt;More training data&lt;/li&gt;
      &lt;li&gt;Reduce the number of features&lt;/li&gt;
      &lt;li&gt;Regularization
        &lt;ul&gt;
          &lt;li&gt;Let’s not have too big numbers in the weight
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/008.JPG&quot; alt=&quot;008&quot; /&gt;&lt;/li&gt;
        &lt;/ul&gt;
      &lt;/li&gt;
      &lt;li&gt;
        Dropout: A Simple Way to Prevent Neural Networks from Overfitting [Srivastava et al. 2014]
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/009.JPG&quot; alt=&quot;009&quot; /&gt;
      &lt;/li&gt;
      &lt;li&gt;
        Regularization: Dropout “randomly set some neurons to zero in the forward pass”
      &lt;/li&gt;
      &lt;li&gt;dropout TensorFlow implementation&lt;/li&gt;
    &lt;/ul&gt;

    &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;dropout_rate = tf.placeholder(&quot;float&quot;)
_L1 = tf.nn.relu(tf.add(tf.matmul(X, W1), B1))
L1 = tf.nn.dropout(_L1, dropout_rate)
&lt;/code&gt;&lt;/pre&gt;
    &lt;/div&gt;

    &lt;blockquote&gt;
      &lt;ul&gt;
        &lt;li&gt;TRAIN:
          &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;sess.run(optimizer, feed_dict={X: batch_xs, Y: batch_ys, dropout_rate: 0.7})
&lt;/code&gt;&lt;/pre&gt;
          &lt;/div&gt;
        &lt;/li&gt;
      &lt;/ul&gt;
    &lt;/blockquote&gt;

    &lt;blockquote&gt;
      &lt;ul&gt;
        &lt;li&gt;EVALUATION:
          &lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;print &quot;Accuracy:&quot;, accuracy.eval({X: mnist.test.images, Y:mnist.test.labels, dropout_rate: 1}) 
&lt;/code&gt;&lt;/pre&gt;
          &lt;/div&gt;
        &lt;/li&gt;
      &lt;/ul&gt;
    &lt;/blockquote&gt;
  &lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;what-is-the-ensemble-&quot;&gt;What is the ensemble ?&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-02/010.JPG&quot; alt=&quot;010&quot; /&gt;

</description>
        <pubDate>Sun, 15 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-02/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-02/</guid>
        
        <category>Deep-Learning</category>
        
        <category>summary</category>
        
        <category>hunkim</category>
        
        <category>github</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>Deep Learning summary from http://hunkim.github.io/ml/ (1)</title>
        <description>&lt;h2 id=&quot;simplified-hypothesis-and-cost&quot;&gt;Simplified Hypothesis and Cost&lt;/h2&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/001.JPG&quot; alt=&quot;001&quot; /&gt;

&lt;h4 id=&quot;gradient-descent-algorithm&quot;&gt;Gradient descent algorithm&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;Minimize cost function&lt;/li&gt;
  &lt;li&gt;Gradient descent is used many minimization problems&lt;/li&gt;
  &lt;li&gt;For a given cost function, cost (W, b), it will find W, b to minimize cost&lt;/li&gt;
  &lt;li&gt;It can be applied to more general function: cost (w1, w2, …)&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;how-to-work&quot;&gt;How to work&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;Formal definition&lt;/li&gt;
&lt;/ul&gt;

&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/002.JPG&quot; alt=&quot;002&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/003.JPG&quot; alt=&quot;003&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/004.JPG&quot; alt=&quot;004&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/005.JPG&quot; alt=&quot;005&quot; /&gt;

&lt;h4 id=&quot;hypothesis-with-matrix&quot;&gt;Hypothesis with Matrix&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/006.JPG&quot; alt=&quot;006&quot; /&gt;

&lt;h2 id=&quot;logistic-regression-classification&quot;&gt;Logistic (regression) classification&lt;/h2&gt;

&lt;h4 id=&quot;logistic-hypothesis&quot;&gt;Logistic Hypothesis&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/007.JPG&quot; alt=&quot;007&quot; /&gt;

&lt;h4 id=&quot;new-cost-function-for-logistic&quot;&gt;New cost function for logistic&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/008.JPG&quot; alt=&quot;008&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/009.JPG&quot; alt=&quot;009&quot; /&gt;

&lt;strong&gt;sample code&lt;/strong&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;h = tf.matmul(W, X)
hypothesis = tf.div(1., 1. + tf.exp(-h))

cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))

a = tf.Variable(0.1)  # learning rate, alpha
optimizer = tf.train.GradientDescentOptimizer(a)
train = optimizer.minimize(cost)  # goal is minimize cost
&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;

&lt;h4 id=&quot;multinomial-classification-and-where-is-sigmoid&quot;&gt;Multinomial classification and Where is sigmoid?&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/010.JPG&quot; alt=&quot;010&quot; /&gt;

&lt;h4 id=&quot;softmax&quot;&gt;SOFTMAX&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/011.JPG&quot; alt=&quot;011&quot; /&gt;

&lt;h4 id=&quot;cross-entropy-cost-function&quot;&gt;Cross-entropy cost function&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/012.JPG&quot; alt=&quot;012&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/013.JPG&quot; alt=&quot;013&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/014.JPG&quot; alt=&quot;014&quot; /&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/015.JPG&quot; alt=&quot;015&quot; /&gt;

&lt;h4 id=&quot;data-x-preprocessing-for-gradient-descent&quot;&gt;Data (X) preprocessing for gradient descent&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/016.JPG&quot; alt=&quot;016&quot; /&gt;

&lt;h4 id=&quot;standardization&quot;&gt;standardization&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/017.JPG&quot; alt=&quot;017&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;X_std[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;training-validation-and-test-sets&quot;&gt;Training, validation and test sets&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/018.JPG&quot; alt=&quot;018&quot; /&gt;

&lt;h4 id=&quot;minist-dataset&quot;&gt;MINIST Dataset&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-15-Deep-Learning-hunkim-github-01/019.JPG&quot; alt=&quot;019&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;train-images-idx3-ubyte.gz : training set images (9912422 bytes)&lt;/li&gt;
  &lt;li&gt;
    train-labels-idx1-ubyte.gz : training set lables (18881 bytes)
  &lt;/li&gt;
  &lt;li&gt;t10k-images-idx3-ubyte.gz : test set images (1648877 bytes)&lt;/li&gt;
  &lt;li&gt;
    t10k-labels-idx1-ubyte.gz : test set lables ( 4542 bytes)
  &lt;/li&gt;
  &lt;li&gt;&lt;a href=&quot;http://yann.lecun.com/exdb/mnist/&quot;&gt;http://yann.lecun.com/exdb/mnist/&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
        <pubDate>Sun, 15 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-01/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/15/Deep-Learning-hunkim-github-01/</guid>
        
        <category>Deep-Learning</category>
        
        <category>summary</category>
        
        <category>hunkim</category>
        
        <category>github</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>Summary of CNN base cs231n.github.io  </title>
        <description>CNN Base 인 cs231n을 정리하기
</description>
        <pubDate>Sun, 15 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/15/Deep-Learning-CNN-cs231n-github/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/15/Deep-Learning-CNN-cs231n-github/</guid>
        
        <category>Deep-Learning</category>
        
        <category>summary</category>
        
        <category>cs231n</category>
        
        <category>github</category>
        
        
        <category>Deep-Learning</category>
        
      </item>
    
      <item>
        <title>BOOK-TCP/IP 쉽게 더 쉽게-03</title>
        <description>&lt;h1 id=&quot;보안-프로토콜-적용하기&quot;&gt;보안 프로토콜 적용하기&lt;/h1&gt;
&lt;ul&gt;
  &lt;li&gt;HTTPS : HTTP + SSL/TLS을 적용한 것&lt;/li&gt;
  &lt;li&gt;VPN : 통신 거점간의 통신을 통째로 암호화하는 방식하므로 app별로 암호화하지 않아도 된다.&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;공유키--하나의-키로-암호화복호화가-가능한-방법&quot;&gt;공유키 : 하나의 키로 암호화/복호화가 가능한 방법&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;키를 공유할 모든 사람에게 공유해야 하므로 부담이 되는 방법&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;공개키--암호화키와-복호화키를-세트로-관리&quot;&gt;공개키 : 암호화키와 복호화키를 세트로 관리&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;개인키(비밀키) : 자신이 가지는 키&lt;/li&gt;
  &lt;li&gt;공개키 : 상대방에게 전달하는 키.&lt;/li&gt;
  &lt;li&gt;공개키와 개인키의 사용절차
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/001.JPG&quot; alt=&quot;001&quot; /&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;전자인증서--사용자가-본인이라는-것을-증명&quot;&gt;전자인증서 : 사용자가 본인이라는 것을 증명&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/002.JPG&quot; alt=&quot;002&quot; /&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/003.JPG&quot; alt=&quot;003&quot; /&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/004.JPG&quot; alt=&quot;004&quot; /&gt;

&lt;ul&gt;
  &lt;li&gt;PKI ( Public Key Infrastructure )&lt;/li&gt;
  &lt;li&gt;CA ( Certification Authority ) : 한국정보인증, 코스콤, 한국전자인증, 한국무역정보통신, 금융결재원&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;전자서명---데이터가-위변조-되지-않았다는-것을-증명&quot;&gt;전자서명  : 데이터가 위변조 되지 않았다는 것을 증명&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;반드시 개인키로 암호화 한다.&lt;/li&gt;
  &lt;li&gt;개인키로 암호화한 데이터는 공개키로 복호화할 수 있다
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/005.JPG&quot; alt=&quot;005&quot; /&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/006.JPG&quot; alt=&quot;006&quot; /&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;이메일을-안전하게-전달--smime&quot;&gt;이메일을 안전하게 전달 : S/MIME&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;secure /Multipurpose Internet Mail Extenstion&lt;/li&gt;
  &lt;li&gt;
    이메일을 암복호화 할때는 공개키와 공유키를 조합 사용한다. 
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/007.JPG&quot; alt=&quot;007&quot; /&gt;
  &lt;/li&gt;
  &lt;li&gt;공유키 방식을 조합하는 이유 :&lt;/li&gt;
  &lt;li&gt;이메일 본문을 공개키 방식으로 암복호화하는 것보다 공유키 방식으로 암복호화하는 것이 연산처리에서 부담이 적다&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;ssl--tls&quot;&gt;SSL / TLS&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;
    SSL ( Secure Sockets Layer), TLS(Transport Layer Secure)
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/008.JPG&quot; alt=&quot;008&quot; /&gt;
  &lt;/li&gt;
  &lt;li&gt;
    HTTPS : SSL/TLS을 사용한다.
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/009.JPG&quot; alt=&quot;009&quot; /&gt;
  &lt;/li&gt;
  &lt;li&gt;SSL은 넷스케이프 커뮤니케이션즈가 개발하였고, IETF(Internet Engineering Task Force)는 이것을 TLS 로 퓨준화 후에 기존의 SSL을 대체하였다.&lt;/li&gt;
  &lt;li&gt;TSL 대신 SSL이라고 부르는 경우가 많다.&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;ssh--secure-shell-&quot;&gt;SSH ( Secure Shell )&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;원격지의 컴퓨터를 제어하기 위해&lt;/li&gt;
  &lt;li&gt;공개키와 공유키를 조합하여 사용.&lt;/li&gt;
  &lt;li&gt;전자인증서를 사용하지 않는다.
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/010.JPG&quot; alt=&quot;010&quot; /&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/011.JPG&quot; alt=&quot;011&quot; /&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h4 id=&quot;방화벽&quot;&gt;방화벽&lt;/h4&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/012.JPG&quot; alt=&quot;012&quot; /&gt;

&lt;h4 id=&quot;무선랜-보안&quot;&gt;무선랜 보안&lt;/h4&gt;
&lt;ul&gt;
  &lt;li&gt;
    WEP(Wired Equipment Privacy), WPA(Wi-fi Protected Access)
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/013.JPG&quot; alt=&quot;013&quot; /&gt;
  &lt;/li&gt;
  &lt;li&gt;암복호화 하기 위해 공유키를 사용한다.&lt;/li&gt;
  &lt;li&gt;WPA2에서는 블록 암호화 방식을 사용한다
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/014.JPG&quot; alt=&quot;014&quot; /&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h5 id=&quot;패스프레이즈&quot;&gt;패스프레이즈&lt;/h5&gt;
&lt;ul&gt;
  &lt;li&gt;키 = 패스프레이즈(사용자가 지정한 문자열) + 초기화벡터(조금씩 변화는 값)&lt;/li&gt;
  &lt;li&gt;암호화는 쉽고, 복호화는 어렵다
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/015.JPG&quot; alt=&quot;015&quot; /&gt;
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/016.JPG&quot; alt=&quot;016&quot; /&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3 id=&quot;vpn&quot;&gt;VPN&lt;/h3&gt;
&lt;ul&gt;
  &lt;li&gt;
    Virtual Private Network 
&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/017.JPG&quot; alt=&quot;017&quot; /&gt;
  &lt;/li&gt;
  &lt;li&gt;IPSec(Security Architecture for Internet Protocol)&lt;/li&gt;
  &lt;li&gt;PPTP(Point to Point Tunneling Protocol)&lt;/li&gt;
&lt;/ul&gt;

&lt;img src=&quot;/img/2017-01-14-CCNA-book-TCP-IP-easy-and-easy-03/018.JPG&quot; alt=&quot;018&quot; /&gt;

</description>
        <pubDate>Sat, 14 Jan 2017 00:00:00 +0000</pubDate>
        <link>https://jaehyek.github.io/2017/01/14/CCNA-book-TCP-IP-easy-and-easy-03/</link>
        <guid isPermaLink="true">https://jaehyek.github.io/2017/01/14/CCNA-book-TCP-IP-easy-and-easy-03/</guid>
        
        <category>TCP-IP</category>
        
        
        <category>CCNA</category>
        
      </item>
    
  </channel>
</rss>
