atom feed1 message in org.apache.spark.issues[jira] [Commented] (SPARK-11035) Laun...
FromSent OnAttachments
Apache Spark (JIRA)Oct 27, 2017 3:49 pm 
Subject:[jira] [Commented] (SPARK-11035) Launcher: allow apps to be launched in-process
From:Apache Spark (JIRA) (
Date:Oct 27, 2017 3:49:00 pm


Apache Spark commented on SPARK-11035:

User 'vanzin' has created a pull request for this issue:

Launcher: allow apps to be launched in-process


Key: SPARK-11035 URL: Project: Spark Issue Type: Sub-task Components: Spark Core Affects Versions: 1.6.0 Reporter: Marcelo Vanzin

The launcher library is currently restricted to launching apps as child
processes. That is fine for a lot of cases, especially if the app is running in
client mode. But in certain cases, especially launching in cluster mode, it's more efficient
to avoid launching a new process, since that process won't be doing much. We should add support for launching apps in process, even if restricted to
cluster mode at first. This will require some rework of the launch paths to
avoid using system properties to propagate configuration.