Add new post about eddl part 2

_site/archive.html

@@ -428,7 +428,7 @@ c13 9 26 20 30 26 7 11 -9 26 -27 26 -5 0 -3 -5 5 -10 9 -6 10 -10 3 -10 -24
   <ul class="menu">
     <li>
       <button type="button" class="button button--secondary button--pill tag-button tag-button--all" data-encode="">
-        Show All<div class="tag-button__count">6</div>
+        Show All<div class="tag-button__count">7</div>
       </button>
     </li><li><button type="button" class="button button--pill tag-button tag-button-1" data-encode="Network">
           <span>Network</span><div class="tag-button__count">1</div>
@@ -436,8 +436,8 @@ c13 9 26 20 30 26 7 11 -9 26 -27 26 -5 0 -3 -5 5 -10 9 -6 10 -10 3 -10 -24
       </li><li><button type="button" class="button button--pill tag-button tag-button-1" data-encode="Nonsense">
           <span>Nonsense</span><div class="tag-button__count">1</div>
         </button>
-      </li><li><button type="button" class="button button--pill tag-button tag-button-2" data-encode="Research">
-          <span>Research</span><div class="tag-button__count">2</div>
+      </li><li><button type="button" class="button button--pill tag-button tag-button-3" data-encode="Research">
+          <span>Research</span><div class="tag-button__count">3</div>
         </button>
       </li><li><button type="button" class="button button--pill tag-button tag-button-1" data-encode="visualization">
           <span>visualization</span><div class="tag-button__count">1</div>
@@ -448,6 +448,8 @@ c13 9 26 20 30 26 7 11 -9 26 -27 26 -5 0 -3 -5 5 -10 9 -6 10 -10 3 -10 -24
       </li></ul>
 </div>
 <div class="js-result layout--archive__result d-none"><div class="article-list items"><section><h2 class="article-list__group-header">2021</h2><ul class="items"><li class="item" itemscope itemtype="http://schema.org/BlogPosting" data-tags="Research">
+        <div class="item__content"><span class="item__meta">Oct 31</span><a itemprop="headline" class="item__header" href="/posts/eddl-how-do-we-train-on-limited-edge-devices-part2">EDDL: How do we train neural networks on limited edge devices - PART 2</a></div>
+      </li><li class="item" itemscope itemtype="http://schema.org/BlogPosting" data-tags="Research">
         <div class="item__content"><span class="item__meta">Oct 13</span><a itemprop="headline" class="item__header" href="/posts/eddl-how-do-we-train-on-limited-edge-devices">EDDL: How do we train neural networks on limited edge devices - PART 1</a></div>
       </li></ul></section><section><h2 class="article-list__group-header">2020</h2><ul class="items"><li class="item" itemscope itemtype="http://schema.org/BlogPosting" data-tags="visualization">
         <div class="item__content"><span class="item__meta">Sep 15</span><a itemprop="headline" class="item__header" href="/posts/generate-word-cloud-with-chinese-fenci">Generate Word Cloud Figures with Chinese-Tokenization and WordCloud python libraries</a></div>

_site/assets/search.js

@@ -1 +1 @@
-window.TEXT_SEARCH_DATA={'posts':[{'title':"Stop Talking is the worst title of one blog",'url':"/posts/welcome-to-my-blog"},{'title':"Using charles proxy to monitor mobile SSL traffics",'url':"/posts/charles-is-not-a-good-tool"},{'title':"Some of my previews experiment works: 2016",'url':"/posts/some-of-my-previews-exper-work"},{'title':"Xv6 introduction",'url':"/posts/intro-xv6"},{'title':"Generate Word Cloud Figures with Chinese-Tokenization and WordCloud python libraries",'url':"/posts/generate-word-cloud-with-chinese-fenci"},{'title':"EDDL: How do we train neural networks on limited edge devices - PART 1",'url':"/posts/eddl-how-do-we-train-on-limited-edge-devices"}]};
+window.TEXT_SEARCH_DATA={'posts':[{'title':"Stop Talking is the worst title of one blog",'url':"/posts/welcome-to-my-blog"},{'title':"Using charles proxy to monitor mobile SSL traffics",'url':"/posts/charles-is-not-a-good-tool"},{'title':"Some of my previews experiment works: 2016",'url':"/posts/some-of-my-previews-exper-work"},{'title':"Xv6 introduction",'url':"/posts/intro-xv6"},{'title':"Generate Word Cloud Figures with Chinese-Tokenization and WordCloud python libraries",'url':"/posts/generate-word-cloud-with-chinese-fenci"},{'title':"EDDL: How do we train neural networks on limited edge devices - PART 1",'url':"/posts/eddl-how-do-we-train-on-limited-edge-devices"},{'title':"EDDL: How do we train neural networks on limited edge devices - PART 2",'url':"/posts/eddl-how-do-we-train-on-limited-edge-devices-part2"}]};

_site/index.html

@@ -424,7 +424,21 @@ c13 9 26 20 30 26 7 11 -9 26 -27 26 -5 0 -3 -5 5 -10 9 -6 10 -10 3 -10 -24
               <div class="col-main cell cell--auto"><!-- start custom main top snippet -->
 
 <!-- end custom main top snippet -->
-<article itemscope itemtype="http://schema.org/WebPage"><header style="display:none;"><h1>Home</h1></header><meta itemprop="headline" content="Home"><meta itemprop="author" content="Pengzhan Hao"/><div class="js-article-content"><div class="layout--articles"><div class="article-list items items--divided"><article class="item" itemscope itemtype="http://schema.org/BlogPosting"><div class="item__image" style="vertical-align: middle"><img class="image" src="/static/2021-10/edgelearn-1.png" /></div><div class="item__content">
+<article itemscope itemtype="http://schema.org/WebPage"><header style="display:none;"><h1>Home</h1></header><meta itemprop="headline" content="Home"><meta itemprop="author" content="Pengzhan Hao"/><div class="js-article-content"><div class="layout--articles"><div class="article-list items items--divided"><article class="item" itemscope itemtype="http://schema.org/BlogPosting"><div class="item__image" style="vertical-align: middle"><img class="image" src="/static/2021-10/f.5_Impl_leader_worker.png" /></div><div class="item__content">
+          <header><a href="/posts/eddl-how-do-we-train-on-limited-edge-devices-part2"><h2 itemprop="headline" class="item__header">EDDL: How do we train neural networks on limited edge devices - PART 2</h2></a></header>
+          <div class="item__description"><div class="article__content" itemprop="description articleBody">In the last post, part1, our idea of distributed learning on edge environment was generally addressed.
+I introduced the reason why edge distributed learning is needed and what improvements it can achieve.
+In this post, I will talk about our motivation study and how our framework works.
+
+How does data support us training on edge?
+
+Before designin...</div><p><a href="/posts/eddl-how-do-we-train-on-limited-edge-devices-part2">Read more</a></p></div><div class="article__info clearfix"><ul class="left-col menu"><li>
+              <a class="button button--secondary button--pill button--sm"
+                href="/archive.html?tag=Research">Research</a>
+            </li></ul><ul class="right-col menu"><li><i class="fas fa-user"></i> <span>Pengzhan Hao</span></li><li><i class="far fa-calendar-alt"></i> <span>Oct 31, 2021</span>
+            </li></ul></div><meta itemprop="author" content="Pengzhan Hao"/><meta itemprop="datePublished" content="2021-10-31T13:01:14-04:00">
+    <meta itemprop="keywords" content="Research"></div>
+      </article><article class="item" itemscope itemtype="http://schema.org/BlogPosting"><div class="item__image" style="vertical-align: middle"><img class="image" src="/static/2021-10/edgelearn-1.png" /></div><div class="item__content">
           <header><a href="/posts/eddl-how-do-we-train-on-limited-edge-devices"><h2 itemprop="headline" class="item__header">EDDL: How do we train neural networks on limited edge devices - PART 1</h2></a></header>
           <div class="item__description"><div class="article__content" itemprop="description articleBody">This post introduces our previous milestone in project “Edge trainer”, as the paper “EDDL: A Distributed Deep Learning System for Resource-limited Edge Computing Environment.” was published.
 As the first part of the introductions, I focus only on the motivation and summary of our works.
@@ -444,7 +458,7 @@ If your written language is based on latin alphabet(or other language has space
             </li></ul><ul class="right-col menu"><li><i class="fas fa-user"></i> <span>Pengzhan Hao</span></li><li><i class="far fa-calendar-alt"></i> <span>Sep 15, 2020</span>
             </li></ul></div><meta itemprop="author" content="Pengzhan Hao"/><meta itemprop="datePublished" content="2020-09-15T22:00:14-04:00">
     <meta itemprop="keywords" content="visualization"></div>
-      </article><article class="item" itemscope itemtype="http://schema.org/BlogPosting"><div class="item__content">
+      </article><article class="item" itemscope itemtype="http://schema.org/BlogPosting"><div class="item__image" style="vertical-align: middle"><img class="image" src="/static/2021-10/Xv6_LS_Command_Output.png" /></div><div class="item__content">
           <header><a href="/posts/intro-xv6"><h2 itemprop="headline" class="item__header">Xv6 introduction</h2></a></header>
           <div class="item__description"><div class="article__content" itemprop="description articleBody">In this post, you will learn a few basic concepts of xv6. Learning path will be closed coupled to first project assignment I gave when I assisted in teaching OS classes.
 Understand system call and know how to implement a simple one will be coved as the first half.

_site/posts/eddl-how-do-we-train-on-limited-edge-devices.html

@@ -441,42 +441,42 @@ More details in design and implementation can be found in late posts.</p>
 
 <h2 id="why-do-we-need-training-on-edge">Why do we need training on edge?</h2>
 
-<p>Cloud is not trustworthy anymore. More and more facts supports that breach on cloud happens frequently than before.
-Nowadays, with more generated personal sensitive data has been uploaded to the cloud center, tech company know better to someones than user themselves.</p>
+<p>Cloud is not trustworthy anymore. More and more facts support that breach on the cloud happens frequently than before.
+Nowadays, with more generated personal sensitive data has been uploaded to the cloud center, tech companies know better to someones than the user.</p>
 
-<p>Researchers, no matter in industry on academia, are working in a way that still learning from users’ data but also keeping raw sensitive data under users’ control.
-Many publications already showed feasibility of only sharing after-trained model instead of raw data.
+<p>Researchers, no matter in the industry on academia, are working in a way that still learning from users’ data but also keeping raw sensitive data under users’ control.
+Many publications have already shown the feasibility of only sharing the after-trained model instead of raw data.
 One recent popular study on this is google’s <a href="https://ai.googleblog.com/2017/04/federated-learning-collaborative.html">federated learning</a>.</p>
 
-<p>During investigated this problem, we found that let end user train their own data is safe, but sacrifice efficiency.
+<p>During investigating this problem, we found that letting end-user train their data is safe, but sacrifice efficiency.
 Since one end device has limited resources, training time and power consumption can be disappointing.
-We believe there must have a leverage between privacy and efficiency in some target scenarios.</p>
+We believe there must have leverage between privacy and efficiency in some target scenarios.</p>
 
-<p>Fortunately, we observed that users who belongs to the same campus, plant, firm and community always share similar interests.
+<p>Fortunately, we observed that users who belong to the same campus, plant, firm, and community always share similar interests.
 Therefore, these co-located users have similar demands in using AI-involved routines.
-Also, co-located users are easily targeted by same type of threats, such as ransomware to financial practitioners.</p>
+Also, co-located users are easily targeted by the same type of threats, such as ransomware to financial practitioners.</p>
 
-<p>Think about this, sending features of a new malware app to cloud services in order to train a neural networks used by antivirus program.
-This process may takes long time and small amount of samples may not be recognized by the global neural networks model.
-With a customized local model trained and deployed on the edge can successfully counter the problem.
-With edge training as a supplement of cloud training can achieve better response time and let the whole system more flexible.</p>
+<p>Think about this, sending features of a new malware app to cloud services to train neural networks used by antivirus programs.
+This process may take a long time and a small number of samples may not be recognized by the global neural networks model.
+A customized local model trained and deployed on the edge can successfully counter the problem.
+With edge training as a supplement to the cloud training can achieve better response time and let the whole system more flexible.</p>
 
 <h2 id="why-training-on-edge-is-hard">Why training on edge is hard?</h2>
 
-<p>Since all co-located users’ device can be used for an edge training, issues and challenges occur as deploying this distributed system.</p>
+<p>Since all co-located users’ devices can be used for edge training, issues and challenges occur as deploying this distributed system.</p>
 
 <p>The first challenge is <strong>struggling workers</strong>.
-Training devices are heterogeneity, from limited IoT camera to high-end media center with powerful GPU.
-They are not designed to do machine learnings.
-So, a good edge-based distributed learning framework must can handle variety speeds in training tasks.</p>
+Training devices are heterogeneous, from limited IoT cameras to high-end media centers with powerful GPUs.
+They are not designed to do machine learning.
+So, a good edge-based distributed learning framework must be able to handle a variety of speeds in training tasks.</p>
 
 <p>The second challenge is how to <strong>scale up</strong> clusters.
-In a campus, thousands and more devices may contribute computing resources to the same training tasks.
-However, these devices may located in far not matter in physical or in network topology. 
-How can we well use them well, without struggled with endless transmission time remains a challenge.</p>
+On a campus, thousands and more devices may contribute computing resources to the same training tasks.
+However, these devices may be located far no matter in physical or in network topology.
+The question of how can we well use them well, without struggling with endless transmission time remains a challenge.</p>
 
 <p>The third issue is frequently <strong>joining and exiting</strong> of devices.
-We can’t rely on each devices to faithfully working on training tasks rather than their original workload.
+We can’t rely on each device to faithfully work on training tasks rather than their original workload.
 Smartly schedule work balance and handle join/exit issues also need under consideration.</p>
 
 <h2 id="our-proposal">Our proposal</h2>
@@ -485,29 +485,29 @@ Smartly schedule work balance and handle join/exit issues also need under consid
   <li>
     <p>Dynamic training data distribution and runtime profiler</p>
 
-    <p>We design a dynamic training data distribution mechanism that helps to both the first and the third challenges.
-  Preprocessing data can be transmitted without leakage of raw sensitive information. 
-  This can helps with struggling workers who can train small batches in order to upload parameters with a similar training time.
-  Also, for extremely slow devices, join and exit of devices cases, dynamic data distribution and profiler can helps with keep global training parameters from polluted and staleness.</p>
+    <p>We design a dynamic training data distribution mechanism that helps both the first and the third challenges.
+  Preprocessing data can be transmitted without leakage of raw and sensitive information.
+  This can help struggling workers who can train small batches in order to upload parameters with a similar training time.
+  Also, for extremely slow devices, join and exit of devices cases, dynamic data distribution and profiler can help with keeping global training parameters from pollution and staleness.</p>
 
-    <p>To counter heterogeneity’s, more approaches were applied in our later research.
-  More details were introduced to runtime profiler in the later works.</p>
+    <p>To counter heterogeneity, more approaches were applied in our later research.
+  More details were introduced to the runtime profiler in the later works.</p>
   </li>
   <li>
     <p>Asynchronous and synchronous aggregation enabled</p>
 
-    <p>In our findings, asynchronous and synchronous parameter update have their pros and cons. 
-  Keeping sync all the time leads struggling worker issue unsolvable.
-  However, async’s harm to accuracy and convergence time also need attentions.
-  To carefully chose between these two update policies at the runtime is what we proposed to make use of their own advantages.</p>
+    <p>In our findings, asynchronous and synchronous parameter update have their pros and cons.
+  Keeping sync all the time leads to struggling worker issues unsolvable.
+  However, async’s harm to accuracy and convergence time also needs attention.
+  To carefully choose between these two update policies at the runtime is what we proposed to make use of their own advantages.</p>
   </li>
   <li>
     <p>Leader role splitting</p>
 
-    <p>The idea is to let worker devices with higher bandwidth taking leader role during training.
-  Parameter updating does not require much computation but only need bandwidth. 
+    <p>The idea is to let worker devices with higher bandwidth take leader-role during training.
+  Parameter updating does not require much computation but only needs a great of bandwidth.
   Devices with sufficient bandwidth can also work as virtual leader devices.
-  This approach helps with minimize physical devices we used and more leaders can further scale up workers limits.</p>
+  This approach helps minimize physical devices we used and more leaders can further scale up workers’ limits.</p>
   </li>
 </ul>
 </div><section class="article__sharing d-print-none"></section><div class="d-print-none"><footer class="article__footer"><meta itemprop="dateModified" content="2021-10-13T16:53:20-04:00"><!-- start custom article footer snippet -->
@@ -515,7 +515,7 @@ Smartly schedule work balance and handle join/exit issues also need under consid
 <!-- end custom article footer snippet -->
 <div class="article__subscribe"><div class="subscribe"><i class="fas fa-rss"></i> <a type="application/rss+xml" href="/feed.xml">Subscribe</a></div>
 </div><div class="article__license"></div></footer>
-<div class="article__section-navigator clearfix"><div class="previous"><span>PREVIOUS</span><a href="/posts/generate-word-cloud-with-chinese-fenci">Generate Word Cloud Figures with Chinese-Tokenization and WordCloud python libraries</a></div></div></div>
+<div class="article__section-navigator clearfix"><div class="previous"><span>PREVIOUS</span><a href="/posts/generate-word-cloud-with-chinese-fenci">Generate Word Cloud Figures with Chinese-Tokenization and WordCloud python libraries</a></div><div class="next"><span>NEXT</span><a href="/posts/eddl-how-do-we-train-on-limited-edge-devices-part2">EDDL: How do we train neural networks on limited edge devices - PART 2</a></div></div></div>
 
 </div>
 

_site/sitemap.xml

@@ -25,6 +25,10 @@
 <lastmod>2021-10-13T16:53:20-04:00</lastmod>
 </url>
 <url>
+<loc>https://codersherlock.github.com//posts/eddl-how-do-we-train-on-limited-edge-devices-part2</loc>
+<lastmod>2021-10-31T13:01:14-04:00</lastmod>
+</url>
+<url>
 <loc>https://codersherlock.github.com//about.html</loc>
 </url>
 <url>