Data Analysis using Spark SQL

Skillsoft issued completion badges are earned based on viewing the percentage required or receiving a passing score when assessment is required. Continue to explore Apache Spark, the de facto big data science framework, in this Skillsoft Aspire course. You will learn how to analyze a Spark DataFrame by treating it as though it were a relational database table. Learners discover how to create a view from a Spark DataFrame and run SQL queries against it, and how to define and explore data in Windows. Key concepts in this course include different stages involved in optimizing any query or method call on the contents of a Spark DataFrame; how to create views out of a Spark DataFrame's contents and run queries against them; and how to trim and clean a DataFrame before a view is created, as a precursor to running SQL queries. Next, learn how to perform an analysis of data by running different SQL queries; how to configure a DataFrame with an explicitly defined schema; and define what a window is in the context of Spark. Finally, observe how to create and analyze categories of data in a data set by using Windows.