Tag: spark

  1. Running An Apache Spark Application on Amazon Elastic MapReduce

    This is a series of guided screenshots on how to run an AWS EMR Spark application. Last time we wrote a spark count application that found the list of channels with more than 24 hours of programming. We will run that same application this time on EMR instead of the…

    on Scala hadoop mapreduce spark AWS

  2. Getting Unique Combinations Of Products with Apache Spark

    This Apache Spark snippet looks at users who have rated a series of products and pulls out the unique combinations of the products rated by each user to start building a recommendation system. To get there though we will go through a multi-part MapReduce algorithm. Some Terminology: (k,v) denotes…

    on spark Scala

  3. Running A Count With MapReduce in Apache Spark

    Apache Spark Snippet - Counts This is the first in a series of snippets on Apache Spark programs. In a previous post I ran a machine learning algorithm through Spark and will be following a similar setup using the Hortonworks Sandbox. In the future I'll do some snippets on AWS'…

    on spark Scala mapreduce

  4. Predicting Movie Ratings with Apache Spark, and Hortonworks

    Today's goal is to make a prediction on a movie's rating based on its synopsis using machine learning in an environment that could scale out to hundreds or even thousands of nodes. As the title suggests, I'll be doing it on Apache Spark using MLlib written in Scala. I wanted…

    on spark Scala mapreduce