Mastering Spark with R notes

Using the code provided in section 4.5.1 AND 4.5.2, create an LDA model of the essays in the okc dataset. Create charts of most common terms per topic. ANSWER THE FOLLOWING QUESTIONS: 1. Explain how the html tags and newline characters were removed from the text. 2. How were the individual words combined into complete essays? 3. Explain what an LDA model does. 4. What are stop words? 5. In which 2 topics is the word 'want' not in the top ten? 6. What code snippet causes the topics...

The text is Mastering Spark with R. Using the sc connection that you have built earlier and making sure that you are still using Java 8, install the corr library and execute the following lines of code: >ml_corr(cars) >correlate(cars, use = "", method = "pearson" >correlate(cars, use = "", method = "pearson" %>% shave() %>% rplot() Submit a Word doc with screenshots of the results of running the code. Explain what the chart is showing. Expl...

The text is Mastering Spark with R. Using the sc connection with Java 8, execute the following lines of code: >summarize(cars, mpg_percentile = percentile(mpg, 0.25) >summarize(cars, mpg_percentile = percentile(mpg, 0.25) %>% show_query() >summarize(cars, mpg_percentile = percentile(mpg, array(0.25, 0.5, 0.75) )) >summarize(cars, mpg_percentile = percentile(mpg, array(0.25, 0.5, 0.75) )) %>% mutate(mpg_percentile = explode(mpg_percentile))

The text is Mastering Spark with R. After installing the dplyr library, use the following code in R studio, create a scatter plot of hp v. mpg cars2 <- copy_to(sc, mtcars) cars3 <- (cars2) cars4 <- select(cars3, hp, mpg) cars5 <- sample_n(cars4, 32) cars6 <- collect(cars5) plot(cars6)

Submit a Word or .pdf document with screen shots of R Studio where you have created a vector of three words. Whenever you are asked to submit a screenshot, include either a sliver of your desktop or a timestamp from your desktop. Always repeat the question you are answering.