Docker macvlan and ipvlan network plugins

Sreenivas Makam's Blog

This is a continuation of my previous blog on macvlan and ipvlan Linux network drivers. Docker has added support for macvlan and ipvlan drivers and its currently in experimental mode as of Docker release 1.11.

Example used in this blog

In this example, we will use Docker macvlan and ipvlan network plugins for Container communication across hosts. To illustrate macvlan and ipvlan concepts and usage, I have created the following example.


Following are details of the setup:

  • First, we need to create two Docker hosts with experimental Docker installed. The experimental Docker has support for macvlan and ipvlan. To create experimental boot2docker image, please use the procedure here.
  • It is needed to enable promiscuous mode on the Virtualbox adapter. This allows for Container communication across hosts.
  • There are four Containers in each host. Two Containers are in vlan70 network and two other Containers are in vlan80 network.
  • We will…

View original post 715 more words


Email spam detection using apache spark mllib


In this blog we will see the real use case of spark mllib that is email spam detection. With the help of using the apache spark mllib component we will detect that email will goes in spam folder or primary folder.

So now jump into the programming and see how it will implement. So first we will load the data from training from spam dataset and primary dataset as follow

val spam = sc.textFile("/home/sandy/Spark/enron1/spam/0052.2003-12-20.GP.spam.txt", 4)
val normal = sc.textFile("/home/sandy/Spark/enron1/ham/0022.1999-12-16.farmer.ham.txt", 4)

Next we need to use HashinTF or IDF to find the frequency of word in the mail and create a Vector which is helpful in creating the LabelPoints for the training

val spamFeatures = => tf.transform(email.split(" ")))
val normalFeatures = => tf.transform(email.split(" ")))

With the help of vectors we will create the LabelPoints , LabelPoints are the input for our model we will create label points as follows

View original post 204 more words

Convolutional Neural Networks backpropagation: from intuition to derivation

Grzegorz Gwardys

Disclaimer: It is assumed that the reader is familiar with terms such as Multilayer Perceptron, delta errors or backpropagation. If not,  it is recommended to read for example a chapter 2 of free online book ‘Neural Networks and Deep Learning’ by Michael Nielsen.   

Convolutional Neural Networks (CNN) are now a standard way of image classification – there are publicly accessible deep learning frameworks, trained models and services. It’s more time consuming to install stuff like caffe than to perform state-of-the-art object classification or detection. We also have many methods of getting knowledge -there is a large number of deep learning courses/MOOCs, free e-books or even direct ways of accessing to the strongest Deep/Machine Learning minds such as Yoshua Bengio, Andrew NG or Yann Lecun by Quora, Facebook or G+.

Nevertheless, when I wanted to get deeper insight in CNN, I could not find a “CNN backpropagation for dummies”. Notoriously…

View original post 785 more words