Skip to main content

Big Data & Hadoop

Since everything moves fast in the IT world, you have new terminologies entering their 3rd or 4th generation by the time you get a chance to get your hands dirty with them. Big Data has been one of them, an alluring technology allowing massive distributed power over large datasets using the famous map-reduce algorithm. Apache Hadoop allows scaling to massive proportions and has been in use with tech giants like Google and Facebook.

 I decided to start running a Hadoop cluster myself using the following guide as a started.

 https://github.com/GoogleCloudPlatform/solutions-google-compute-engine-cluster-for-hadoop/blob/master/README.md

This version installs Hadoop locally but uses the Google App Engine and Google Cloud Storage and allows basic scaling/clustering. I started running the pre-requisites on a VM Centos 6.4 and things were going ok. Then I realized that I needed to go deeper into Hadoop and maybe run a sample locally, without achieving the Cloud version first.  Then I went to the following:

http://tecadmin.net/steps-to-install-hadoop-on-centosrhel-6/#

It had simple enough steps to get it installed. Now I am reading Hadoop in Action by Alex Holmes.


Comments

Post a Comment

Popular posts from this blog

Employment Based Green Card Marathon

There are 3 most popular ways to get a Green Card to live and work in US: Through Marriage with a US Citizen, Employment Sponsorship and the Diversity Lottery. I would like to articulate on the unfairness of the process for the Employment based applicants and its repercussions. After getting a scholarship to study at Brandeis University, I arrived in the United States on August 28, 1998 on an F-1 Student Visa. After graduating with my Master's degree, I had 1 year of OPT - Optional Practical Training which allowed me to work for companies that were in fields similar to my concentration. Next chapter in my immigration story is the H1-B Work Visa which is frowned upon. This visa provides an entry point for skilled immigrants and it is one of the very few points of entry to the US based on merit. Scorn on this quota of about 85k is well deserved on an emotional level, especially when considering high unemployment of today's workplace. Yet, working in IT and being involved with...
3 comments
Read more

Windows 7 Do NOT Remember folder settings

There used to be an option in prior windows version where under Folder Options, you would "NOT remember" the view settings. This way, the browsing would be uniform, unless you wanted to change it. I found a way to do it in Win 7. Go to Folder Options\View and click Apply to Folders. This will apply the view in the current window to all folder views.
Post a Comment
Read more

What to do with your Extra Router

When switching to Verizon FIOS, I retired my SMC router. Recently, I was complaining about the weak wireless signal to my HTPC which is on the other side of our condo. Suddenly, it made sense to use the SMC as an extender. Simply, login to the extra routers menu and disable DHCP. Switch mode from Gateway to Repeater (Bridge with some models). Connect a client ethernet cable from your main router to the WAN port of your secondary one. You should be done! Good guides: http://www.tomshardware.com/forum/23864-42-exactly-connect-router-link-first http://ask-leo.com/can_i_use_a_second_wireless_access_point_to_extend_my_wireless_network.html
Post a Comment
Read more