Category Archives: Spark
Easy tutorial on Spark SQL and DataFrames
In this tutorial, you will learn how to load a DataFrame and perform basic operations on DataFrames with both API and SQL. I’m using colab to run the code. First, we need to To download the required tools !apt-get install openjdk-8-jdk-headless -qq > /dev/null !wget -q https://downloads.apache.org/spark/spark-3.1.1/spark-3.1.1-bin-hadoop2.7.tgz !tar -xvf spark-3.1.1-bin-hadoop2.7.tgz !pip install -q findspark Then, […]