最新のAssociate-Developer-Apache-Spark問題集で試験を準備する

Databricks Associate-Developer-Apache-Spark 試験概要:

Databricks Associate-Developer-Apache-Spark 試験シラバストピック:

Databricks Certified Associate Developer for Apache Spark 3.0 認定 Associate-Developer-Apache-Spark 試験問題：

1. Which of the following code blocks returns a DataFrame that matches the multi-column DataFrame itemsDf, except that integer column itemId has been converted into a string column?

A) itemsDf.withColumn("itemId", col("itemId").cast("string"))
(Correct)
B) spark.cast(itemsDf, "itemId", "string")
C) itemsDf.withColumn("itemId", col("itemId").convert("string"))
D) itemsDf.select(cast("itemId", "string"))
E) itemsDf.withColumn("itemId", convert("itemId", "string"))

2. The code block displayed below contains an error. The code block is intended to join DataFrame itemsDf with the larger DataFrame transactionsDf on column itemId. Find the error.
Code block:
transactionsDf.join(itemsDf, "itemId", how="broadcast")

A) Spark will only perform the broadcast operation if this behavior has been enabled on the Spark cluster.
B) The larger DataFrame transactionsDf is being broadcasted, rather than the smaller DataFrame itemsDf.
C) The syntax is wrong, how= should be removed from the code block.
D) The join method should be replaced by the broadcast method.
E) broadcast is not a valid join type.

3. Which of the following code blocks performs an inner join of DataFrames transactionsDf and itemsDf on columns productId and itemId, respectively, excluding columns value and storeId from DataFrame transactionsDf and column attributes from DataFrame itemsDf?

A) 1.transactionsDf.createOrReplaceTempView('transactionsDf')
2.itemsDf.createOrReplaceTempView('itemsDf')
3.
4.spark.sql("SELECT -value, -storeId FROM transactionsDf INNER JOIN itemsDf ON productId==itemId").drop("attributes")
B) 1.transactionsDf \
2. .drop(col('value'), col('storeId')) \
3. .join(itemsDf.drop(col('attributes')), col('productId')==col('itemId'))
C) transactionsDf.drop('value', 'storeId').join(itemsDf.select('attributes'), transactionsDf.productId==itemsDf.itemId)
D) 1.transactionsDf.createOrReplaceTempView('transactionsDf')
2.itemsDf.createOrReplaceTempView('itemsDf')
3.
4.statement = """
5.SELECT * FROM transactionsDf
6.INNER JOIN itemsDf
7.ON transactionsDf.productId==itemsDf.itemId
8."""
9.spark.sql(statement).drop("value", "storeId", "attributes")
E) transactionsDf.drop("value", "storeId").join(itemsDf.drop("attributes"),
"transactionsDf.productId==itemsDf.itemId")

4. The code block displayed below contains an error. The code block should configure Spark to split data in 20 parts when exchanging data between executors for joins or aggregations. Find the error.
Code block:
spark.conf.set(spark.sql.shuffle.partitions, 20)

A) The code block sets the incorrect number of parts.
B) The code block uses the wrong command for setting an option.
C) The code block expresses the option incorrectly.
D) The code block is missing a parameter.
E) The code block sets the wrong option.

5. Which of the following is one of the big performance advantages that Spark has over Hadoop?

A) Spark achieves performance gains for developers by extending Hadoop's DataFrames with a user-friendly API.
B) Spark achieves great performance by storing data in the HDFS format, whereas Hadoop can only use parquet files.
C) Spark achieves higher resiliency for queries since, different from Hadoop, it can be deployed on Kubernetes.
D) Spark achieves great performance by storing data and performing computation in memory, whereas large jobs in Hadoop require a large amount of relatively slow disk I/O operations.
E) Spark achieves great performance by storing data in the DAG format, whereas Hadoop can only use parquet files.

質問と回答：