Skip to main content

Big Data & Hadoop

Since everything moves fast in the IT world, you have new terminologies entering their 3rd or 4th generation by the time you get a chance to get your hands dirty with them. Big Data has been one of them, an alluring technology allowing massive distributed power over large datasets using the famous map-reduce algorithm. Apache Hadoop allows scaling to massive proportions and has been in use with tech giants like Google and Facebook.

I decided to start running a Hadoop cluster myself using the following guide as a started.

This version installs Hadoop locally but uses the Google App Engine and Google Cloud Storage and allows basic scaling/clustering. I started running the pre-requisites on a VM Centos 6.4 and things were going ok. Then I realized that I needed to go deeper into Hadoop and maybe run a sample locally, without achieving the Cloud version first.  Then I went to the following:

It had simple enough steps to get it installed. Now I am reading Hadoop in Action by Alex Holmes.


Popular posts from this blog

The feature you are trying to use is on a network resource that is unavailable

You see this:
The feature you are trying to use is on a network resource that is unavailable. This is happening because the cached installer is missing (in my case due to running a scan on folder sizes, seeing this particular windows folder with all the uninstallers (the cache) and removing them.)

Use this:
Uninstall the program that is causing trouble using the tool and voila.

npm nuget or other repository problems

I was experiencing an issue where my nuget packages would not restore the dlls. I would get Error CS0234 The type or namespace name 'Entity' does not exist in the namespace 'System.Data' (are you missing an assembly reference?) as the error but it would say all nuget packages are already installed. I followed the sage advice of deleting everything in c:\users\\.nuget\packages as well as the packages folder in solution and force a restore. I believe this could be applied to all package repositories in case this type of problems are faced. By the way, check out Artifactory if your environment is hostile to public yum, apt-get, npm, nuget or any other repositories.