7 * When a drush command is called with the --backend option,
8 * it will buffer all output, and instead return a JSON encoded
9 * string containing all relevant information on the command that
12 * Through this mechanism, it is possible for Drush commands to
15 * There are many cases where a command might wish to call another
16 * command in its own process, to allow the calling command to
17 * intercept and act on any errors that may occur in the script that
20 * A simple example is if there exists an 'update' command for running
21 * update.php on a specific site. The original command might download
22 * a newer version of a module for installation on a site, and then
23 * run the update script in a separate process, so that in the case
24 * of an error running a hook_update_n function, the module can revert
25 * to a previously made database backup, and the previously installed code.
27 * By calling the script in a separate process, the calling script is insulated
28 * from any error that occurs in the called script, to the level that if a
29 * php code error occurs (ie: misformed file, missing parenthesis, whatever),
30 * it is still able to reliably handle any problems that occur.
32 * This is nearly a RESTful API. @see http://en.wikipedia.org/wiki/REST
35 * http://[server]/[apipath]/[command]?[arg1]=[value1],[arg2]=[value2]
38 * [apipath] [command] --[arg1]=[value1] --[arg2]=[value2] --backend
40 * [apipath] in this case will be the path to the drush.php file.
41 * [command] is the command you would call, for instance 'status'.
43 * GET parameters will be passed as options to the script.
44 * POST parameters will be passed to the script as a JSON encoded associative array over STDIN.
46 * Because of this standard interface, Drush commands can also be executed on
47 * external servers through SSH pipes, simply by prepending, 'ssh username@server.com'
48 * in front of the command.
50 * If the key-based ssh authentication has been set up between the servers,
51 * this will just work. By default, drush is configured to disallow password
52 * authentication; if you would like to enter a password for every connection,
53 * then in your drushrc.php file, set $options['ssh-options'] so that it does NOT
54 * include '-o PasswordAuthentication=no'. See examples/example.drushrc.php.
56 * The results from backend API calls can be fetched via a call to
57 * drush_backend_get_result().
60 use Drush\Log\LogLevel;
63 * Identify the JSON encoded output from a command.
65 * Note that Drush now outputs a null ("\0") before the DRUSH_BACKEND_OUTPUT_DELIMITER,
66 * but this null occurs where this constant is output rather than being
67 * included in the define. This is done to maintain compatibility with
68 * older versions of Drush, so that Drush-7.x can correctly parse backend messages
69 * from calls made to Drush-5.x and earlier. The null is removed via trim().
71 define('DRUSH_BACKEND_OUTPUT_START', 'DRUSH_BACKEND_OUTPUT_START>>>');
72 define('DRUSH_BACKEND_OUTPUT_DELIMITER', DRUSH_BACKEND_OUTPUT_START . '%s<<<DRUSH_BACKEND_OUTPUT_END');
75 * Identify JSON encoded "packets" embedded inside of backend
76 * output; used to send out-of-band information durring a backend
77 * invoke call (currently only used for log and error messages).
79 define('DRUSH_BACKEND_PACKET_START', "DRUSH_BACKEND:");
80 define('DRUSH_BACKEND_PACKET_PATTERN', "\0" . DRUSH_BACKEND_PACKET_START . "%s\n\0");
83 * The backend result is the original PHP data structure (usually an array)
84 * used to generate the output for the current command.
86 function drush_backend_set_result($value) {
87 if (drush_get_context('DRUSH_BACKEND')) {
88 drush_set_context('BACKEND_RESULT', $value);
93 * Retrieves the results from the last call to backend_invoke.
96 * An associative array containing information from the last
97 * backend invoke. The keys in the array include:
99 * - output: This item contains the textual output of
100 * the command that was executed.
101 * - object: Contains the PHP object representation of the
102 * result of the command.
103 * - self: The self object contains the alias record that was
104 * used to select the bootstrapped site when the command was
106 * - error_status: This item returns the error status for the
107 * command. Zero means "no error".
108 * - log: The log item contains an array of log messages from
109 * the command execution ordered chronologically. Each log
110 * entery is an associative array. A log entry contains
112 * o type: The type of log entry, such as 'notice' or 'warning'
113 * o message: The log message
114 * o timestamp: The time that the message was logged
115 * o memory: Available memory at the time that the message was logged
116 * o error: The error code associated with the log message
117 * (only for log entries whose type is 'error')
118 * - error_log: The error_log item contains another representation
119 * of entries from the log. Only log entries whose 'error' item
120 * is set will appear in the error log. The error log is an
121 * associative array whose key is the error code, and whose value
122 * is an array of messages--one message for every log entry with
123 * the same error code.
124 * - context: The context item contains a representation of all option
125 * values that affected the operation of the command, including both
126 * the command line options, options set in a drushrc.php configuration
127 * files, and options set from the alias record used with the command.
129 function drush_backend_get_result() {
130 return drush_get_context('BACKEND_RESULT');
134 * Print the json-encoded output of this command, including the
135 * encoded log records, context information, etc.
137 function drush_backend_output() {
140 if (drush_get_context('DRUSH_PIPE')) {
141 $pipe = drush_get_context('DRUSH_PIPE_BUFFER');
142 $data['output'] = $pipe; // print_r($pipe, TRUE);
145 // Strip out backend commands.
146 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), array("\0" => "\\0"));
147 $packet_regex = str_replace("\n", "", $packet_regex);
148 $data['output'] = preg_replace("/$packet_regex/s", '', drush_backend_output_collect(NULL));
151 if (drush_get_context('DRUSH_QUIET', FALSE)) {
155 $result_object = drush_backend_get_result();
156 if (isset($result_object)) {
157 $data['object'] = $result_object;
160 $error = drush_get_error();
161 $data['error_status'] = ($error) ? $error : DRUSH_SUCCESS;
163 $data['log'] = drush_get_log(); // Append logging information
164 // The error log is a more specific version of the log, and may be used by calling
165 // scripts to check for specific errors that have occurred.
166 $data['error_log'] = drush_get_error_log();
167 // If there is a @self record, then include it in the result
168 $self_record = drush_sitealias_get_record('@self');
169 if (!empty($self_record)) {
170 $site_context = drush_get_context('site', array());
171 unset($site_context['config-file']);
172 unset($site_context['context-path']);
173 unset($self_record['loaded-config']);
174 unset($self_record['#name']);
175 $data['self'] = array_merge($site_context, $self_record);
178 // Return the options that were set at the end of the process.
179 $data['context'] = drush_get_merged_options();
180 printf("\0" . DRUSH_BACKEND_OUTPUT_DELIMITER, json_encode($data));
184 * Callback to collect backend command output.
186 function drush_backend_output_collect($string) {
188 if (!isset($string)) {
197 * Output buffer functions that discards all output but backend packets.
199 function drush_backend_output_discard($string) {
200 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), array("\0" => "\\0"));
201 $packet_regex = str_replace("\n", "", $packet_regex);
202 if (preg_match_all("/$packet_regex/s", $string, $matches)) {
203 return implode('', $matches[0]);
208 * Output a backend packet if we're running as backend.
211 * The packet to send.
213 * Data for the command.
216 * A boolean indicating whether the command was output.
218 function drush_backend_packet($packet, $data) {
219 if (drush_get_context('DRUSH_BACKEND')) {
220 $data['packet'] = $packet;
221 $data = json_encode($data);
222 // We use 'fwrite' instead of 'drush_print' here because
223 // this backend packet is out-of-band data.
224 fwrite(STDERR, sprintf(DRUSH_BACKEND_PACKET_PATTERN, $data));
232 * Parse output returned from a Drush command.
235 * The output of a drush command
237 * Integrate the errors and log messages from the command into the current process.
239 * Whether output has already been handled.
242 * An associative array containing the data from the external command, or the string parameter if it
243 * could not be parsed successfully.
245 function drush_backend_parse_output($string, $backend_options = array(), $outputted = FALSE) {
246 $regex = sprintf(DRUSH_BACKEND_OUTPUT_DELIMITER, '(.*)');
248 preg_match("/$regex/s", $string, $match);
250 if (!empty($match) && $match[1]) {
251 // we have our JSON encoded string
253 // remove the match we just made and any non printing characters
254 $string = trim(str_replace(sprintf(DRUSH_BACKEND_OUTPUT_DELIMITER, $match[1]), '', $string));
257 if (!empty($output)) {
258 $data = json_decode($output, TRUE);
259 if (is_array($data)) {
260 _drush_backend_integrate($data, $backend_options, $outputted);
268 * Integrate log messages and error statuses into the current
271 * Output produced by the called script will be printed if we didn't print it
272 * on the fly, errors will be set, and log messages will be logged locally, if
273 * not already logged.
276 * The associative array returned from the external command.
278 * Whether output has already been handled.
280 function _drush_backend_integrate($data, $backend_options, $outputted) {
281 // In 'integrate' mode, logs and errors have already been handled
282 // by drush_backend_packet (sender) drush_backend_parse_packets (receiver - us)
283 // during incremental output. We therefore do not need to call drush_set_error
284 // or drush_log here. The exception is if the sender is an older version of
285 // Drush (version 4.x) that does not send backend packets, then we will
286 // not have processed the log entries yet, and must print them here.
287 $received_packets = drush_get_context('DRUSH_RECEIVED_BACKEND_PACKETS', FALSE);
288 if (is_array($data['log']) && $backend_options['log'] && (!$received_packets)) {
289 foreach($data['log'] as $log) {
290 $message = is_array($log['message']) ? implode("\n", $log['message']) : $log['message'];
291 if (isset($backend_options['#output-label'])) {
292 $message = $backend_options['#output-label'] . $message;
294 if (isset($log['error']) && $backend_options['integrate']) {
295 drush_set_error($log['error'], $message);
297 elseif ($backend_options['integrate']) {
298 drush_log($message, $log['type']);
302 // Output will either be printed, or buffered to the drush_backend_output command.
303 // If the output has already been printed, then we do not need to show it again on a failure.
305 if (drush_cmp_error('DRUSH_APPLICATION_ERROR') && !empty($data['output'])) {
306 drush_set_error("DRUSH_APPLICATION_ERROR", dt("Output from failed command :\n !output", array('!output' => $data['output'])));
308 elseif ($backend_options['output']) {
309 _drush_backend_print_output($data['output'], $backend_options);
315 * Supress log message output during backend integrate.
317 function _drush_backend_integrate_log($entry) {
321 * Call an external command using proc_open.
324 * An array of records containing the following elements:
325 * 'cmd' - The command to execute, already properly escaped
326 * 'post-options' - An associative array that will be JSON encoded
327 * and passed to the script being called. Objects are not allowed,
328 * as they do not json_decode gracefully.
329 * 'backend-options' - Options that control the operation of the backend invoke
331 * An array of commands to execute. These commands already need to be properly escaped.
332 * In this case, post-options will default to empty, and a default output label will
335 * An associative array that will be JSON encoded and passed to the script being called.
336 * Objects are not allowed, as they do not json_decode gracefully.
339 * False if the command could not be executed, or did not return any output.
340 * If it executed successfully, it returns an associative array containing the command
341 * called, the output of the command, and the error code of the command.
343 function _drush_backend_proc_open($cmds, $process_limit, $context = NULL) {
344 $descriptorspec = array(
345 0 => array("pipe", "r"), // stdin is a pipe that the child will read from
346 1 => array("pipe", "w"), // stdout is a pipe that the child will write to
349 $open_processes = array();
351 $process_limit = max($process_limit, 1);
352 $is_windows = drush_is_windows();
353 // Loop through processes until they all close, having a nap as needed.
355 while (count($open_processes) || count($cmds)) {
357 if (count($cmds) && (count($open_processes) < $process_limit)) {
358 // Pop the site and command (key / value) from the cmds array
360 list($site, $cmd) = each($cmds);
363 if (is_array($cmd)) {
365 $post_options = $cmd['post-options'];
366 $backend_options = $cmd['backend-options'];
370 $post_options = array();
371 $backend_options = array();
373 $backend_options += array(
374 '#output-label' => '',
375 '#process-read-size' => 4096,
378 drush_log($backend_options['#output-label'] . $c);
379 $process['process'] = proc_open($c, $descriptorspec, $process['pipes'], null, null, array('context' => $context));
380 if (is_resource($process['process'])) {
382 fwrite($process['pipes'][0], json_encode($post_options)); // pass the data array in a JSON encoded string
384 // If we do not close stdin here, then we cause a deadlock;
385 // see: http://drupal.org/node/766080#comment-4309936
386 // If we reimplement interactive commands to also use
387 // _drush_proc_open, then clearly we would need to keep
389 fclose($process['pipes'][0]);
391 $process['info'] = stream_get_meta_data($process['pipes'][1]);
392 stream_set_blocking($process['pipes'][1], FALSE);
393 stream_set_timeout($process['pipes'][1], 1);
394 $bucket[$site]['cmd'] = $c;
395 $bucket[$site]['output'] = '';
396 $bucket[$site]['remainder'] = '';
397 $bucket[$site]['backend-options'] = $backend_options;
398 $bucket[$site]['end_of_output'] = FALSE;
399 $bucket[$site]['outputted'] = FALSE;
400 $open_processes[$site] = $process;
402 // Reset the $nap_time variable as there might be output to process next
406 // Set up to call stream_select(). See:
407 // http://php.net/manual/en/function.stream-select.php
408 // We can't use stream_select on Windows, because it doesn't work for
409 // streams returned by proc_open.
412 $read_streams = array();
413 $write_streams = array();
414 $except_streams = array();
415 foreach ($open_processes as $site => &$current_process) {
416 if (isset($current_process['pipes'][1])) {
417 $read_streams[] = $current_process['pipes'][1];
420 // Wait up to 2s for data to become ready on one of the read streams.
421 if (count($read_streams)) {
422 $ss_result = stream_select($read_streams, $write_streams, $except_streams, 2);
423 // If stream_select returns a error, then fallback to using $nap_time.
424 if ($ss_result !== FALSE) {
430 foreach ($open_processes as $site => &$current_process) {
431 if (isset($current_process['pipes'][1])) {
432 // Collect output from stdout
433 $bucket[$site][1] = '';
434 $info = stream_get_meta_data($current_process['pipes'][1]);
436 if (!feof($current_process['pipes'][1]) && !$info['timed_out']) {
437 $string = $bucket[$site]['remainder'] . fread($current_process['pipes'][1], $backend_options['#process-read-size']);
438 $bucket[$site]['remainder'] = '';
439 $output_end_pos = strpos($string, DRUSH_BACKEND_OUTPUT_START);
440 if ($output_end_pos !== FALSE) {
441 $trailing_string = substr($string, 0, $output_end_pos);
442 $trailing_remainder = '';
443 // If there is any data in the trailing string (characters prior
444 // to the backend output start), then process any backend packets
446 if (strlen($trailing_string) > 0) {
447 drush_backend_parse_packets($trailing_string, $trailing_remainder, $bucket[$site]['backend-options']);
449 // If there is any data remaining in the trailing string after
450 // the backend packets are removed, then print it.
451 if (strlen($trailing_string) > 0) {
452 _drush_backend_print_output($trailing_string . $trailing_remainder, $bucket[$site]['backend-options']);
453 $bucket[$site]['outputted'] = TRUE;
455 $bucket[$site]['end_of_output'] = TRUE;
457 if (!$bucket[$site]['end_of_output']) {
458 drush_backend_parse_packets($string, $bucket[$site]['remainder'], $bucket[$site]['backend-options']);
459 // Pass output through.
460 _drush_backend_print_output($string, $bucket[$site]['backend-options']);
461 if (strlen($string) > 0) {
462 $bucket[$site]['outputted'] = TRUE;
465 $bucket[$site][1] .= $string;
466 $bucket[$site]['output'] .= $string;
467 $info = stream_get_meta_data($current_process['pipes'][1]);
470 // Reset the $nap_time variable as there might be output to process
472 if (strlen($string) > 0) {
477 fclose($current_process['pipes'][1]);
478 unset($current_process['pipes'][1]);
479 // close the pipe , set a marker
481 // Reset the $nap_time variable as there might be output to process
487 // if both pipes are closed for the process, remove it from active loop and add a new process to open.
488 $bucket[$site]['code'] = proc_close($current_process['process']);
489 unset($open_processes[$site]);
491 // Reset the $nap_time variable as there might be output to process next
497 // We should sleep for a bit if we need to, up to a maximum of 1/10 of a
500 usleep(max($nap_time * 500, 100000));
504 // TODO: Handle bad proc handles
512 * Print the output received from a call to backend invoke,
513 * adding the label to the head of each line if necessary.
515 function _drush_backend_print_output($output_string, $backend_options) {
516 if ($backend_options['output'] && !empty($output_string)) {
517 $output_label = array_key_exists('#output-label', $backend_options) ? $backend_options['#output-label'] : FALSE;
518 if (!empty($output_label)) {
519 // Remove one, and only one newline from the end of the
520 // string. Else we'll get an extra 'empty' line.
521 foreach (explode("\n", preg_replace('/\\n$/', '', $output_string)) as $line) {
522 fwrite(STDOUT, $output_label . rtrim($line) . "\n");
526 fwrite(STDOUT, $output_string);
532 * Parse out and remove backend packet from the supplied string and
533 * invoke the commands.
535 function drush_backend_parse_packets(&$string, &$remainder, $backend_options) {
537 $packet_regex = strtr(sprintf(DRUSH_BACKEND_PACKET_PATTERN, "([^\0]*)"), array("\0" => "\\0"));
538 $packet_regex = str_replace("\n", "", $packet_regex);
539 if (preg_match_all("/$packet_regex/s", $string, $match, PREG_PATTERN_ORDER)) {
540 drush_set_context('DRUSH_RECEIVED_BACKEND_PACKETS', TRUE);
541 foreach ($match[1] as $packet_data) {
542 $entry = (array) json_decode($packet_data);
543 if (is_array($entry) && isset($entry['packet'])) {
544 $function = 'drush_backend_packet_' . $entry['packet'];
545 if (function_exists($function)) {
546 $function($entry, $backend_options);
549 drush_log(dt("Unknown backend packet @packet", array('@packet' => $entry['packet'])), LogLevel::NOTICE);
553 drush_log(dt("Malformed backend packet"), LogLevel::ERROR);
554 drush_log(dt("Bad packet: @packet", array('@packet' => print_r($entry, TRUE))), LogLevel::DEBUG);
555 drush_log(dt("String is: @str", array('@str' => $packet_data), LogLevel::DEBUG));
558 $string = preg_replace("/$packet_regex/s", '', $string);
560 // Check to see if there is potentially a partial packet remaining.
561 // We only care about the last null; if there are any nulls prior
562 // to the last one, they would have been removed above if they were
563 // valid drush packets.
564 $embedded_null = strrpos($string, "\0");
565 if ($embedded_null !== FALSE) {
566 // We will consider everything after $embedded_null to be part of
567 // the $remainder string if:
568 // - the embedded null is less than strlen(DRUSH_BACKEND_OUTPUT_START)
569 // from the end of $string (that is, there might be a truncated
570 // backend packet header, or the truncated backend output start
573 // - the embedded null is followed by DRUSH_BACKEND_PACKET_START
574 // (that is, the terminating null for that packet has not been
575 // read into our buffer yet)
576 if (($embedded_null + strlen(DRUSH_BACKEND_OUTPUT_START) >= strlen($string)) || (substr($string, $embedded_null + 1, strlen(DRUSH_BACKEND_PACKET_START)) == DRUSH_BACKEND_PACKET_START)) {
577 $remainder = substr($string, $embedded_null);
578 $string = substr($string, 0, $embedded_null);
584 * Backend command for setting errors.
586 function drush_backend_packet_set_error($data, $backend_options) {
587 if (!$backend_options['integrate']) {
591 if (array_key_exists('#output-label', $backend_options)) {
592 $output_label = $backend_options['#output-label'];
594 drush_set_error($data['error'], $data['message'], $output_label);
598 * Default options for backend_invoke commands.
600 function _drush_backend_adjust_options($site_record, $command, $command_options, $backend_options) {
601 // By default, if the caller does not specify a value for 'output', but does
602 // specify 'integrate' === FALSE, then we will set output to FALSE. Otherwise we
603 // will allow it to default to TRUE.
604 if ((array_key_exists('integrate', $backend_options)) && ($backend_options['integrate'] === FALSE) && (!array_key_exists('output', $backend_options))) {
605 $backend_options['output'] = FALSE;
607 $has_site_specification = array_key_exists('root', $site_record) || array_key_exists('uri', $site_record);
608 $result = $backend_options + array(
614 'dispatch-using-alias' => !$has_site_specification,
616 // Convert '#integrate' et. al. into backend options
617 foreach ($command_options as $key => $value) {
618 if (substr($key,0,1) === '#') {
619 $result[substr($key,1)] = $value;
626 * Execute a new local or remote command in a new process.
628 * n.b. Prefer drush_invoke_process() to this function.
631 * An array of command records to exacute. Each record should contain:
633 * An array containing information used to generate the command.
635 * Optional. A remote host to execute the drush command on.
637 * Optional. Defaults to the current user. If you specify this, you can choose which module to send.
639 * Optional. Defaults to "-o PasswordAuthentication=no"
641 * Optional. An associative array of environmental variables to prefix the Drush command with.
643 * Optional; contains paths to folders and executables useful to the command.
645 * Optional. Defaults to the current drush.php file on the local machine, and
646 * to simply 'drush' (the drush script in the current PATH) on remote servers.
647 * You may also specify a different drush.php script explicitly. You will need
648 * to set this when calling drush on a remote server if 'drush' is not in the
649 * PATH on that machine.
651 * A defined drush command such as 'cron', 'status' or any of the available ones such as 'drush pm'.
653 * An array of arguments for the command.
655 * Optional. An array containing options to pass to the remote script.
656 * Array items with a numeric key are treated as optional arguments to the
659 * Optional. Additional parameters that control the operation of the invoke.
661 * Optional. Defaults to 'GET'.
662 * If this parameter is set to 'POST', the $data array will be passed
663 * to the script being called as a JSON encoded string over the STDIN
664 * pipe of that process. This is preferable if you have to pass
665 * sensitive data such as passwords and the like.
666 * For any other value, the $data array will be collapsed down into a
667 * set of command line options to the script.
669 * Optional. Defaults to TRUE.
670 * If TRUE, any error statuses will be integrated into the current
671 * process. This might not be what you want, if you are writing a
672 * command that operates on multiple sites.
674 * Optional. Defaults to TRUE.
675 * If TRUE, any log messages will be integrated into the current
678 * Optional. Defaults to TRUE.
679 * If TRUE, output from the command will be synchronously printed to
682 * Optional. Defaults to the current drush.php file on the local
683 * machine, and to simply 'drush' (the drush script in the current
684 * PATH) on remote servers. You may also specify a different drush.php
685 * script explicitly. You will need to set this when calling drush on
686 * a remote server if 'drush' is not in the PATH on that machine.
687 * 'dispatch-using-alias'
688 * Optional. Defaults to FALSE.
689 * If specified as a non-empty value the drush command will be
690 * dispatched using the alias name on the command line, instead of
691 * the options from the alias being added to the command line
693 * @param common_options
694 * Optional. Merged in with the options for each invocation.
695 * @param backend_options
696 * Optional. Merged in with the backend options for each invocation.
697 * @param default_command
698 * Optional. Used as the 'command' for any invocation that does not
699 * define a command explicitly.
700 * @param default_site
701 * Optional. Used as the 'site' for any invocation that does not
702 * define a site explicitly.
704 * Optional. Passed in to proc_open if provided.
707 * If the command could not be completed successfully, FALSE.
708 * If the command was completed, this will return an associative array containing the data from drush_backend_output().
710 function drush_backend_invoke_concurrent($invocations, $common_options = array(), $common_backend_options = array(), $default_command = NULL, $default_site = NULL, $context = NULL) {
713 // Slice and dice our options in preparation to build a command string
714 $invocation_options = array();
715 foreach ($invocations as $invocation) {
716 $site_record = isset($invocation['site']) ? $invocation['site'] : $default_site;
717 // NULL is a synonym to '@self', although the latter is preferred.
718 if (!isset($site_record)) {
719 $site_record = '@self';
721 // If the first parameter is not a site alias record,
722 // then presume it is an alias name, and try to look up
724 if (!is_array($site_record)) {
725 $site_record = drush_sitealias_get_record($site_record);
727 $command = isset($invocation['command']) ? $invocation['command'] : $default_command;
728 $args = isset($invocation['args']) ? $invocation['args'] : array();
729 $command_options = isset($invocation['options']) ? $invocation['options'] : array();
730 $backend_options = isset($invocation['backend-options']) ? $invocation['backend-options'] : array();
731 // If $backend_options is passed in as a bool, interpret that as the value for 'integrate'
732 if (!is_array($common_backend_options)) {
733 $integrate = (bool)$common_backend_options;
734 $common_backend_options = array('integrate' => $integrate);
737 $command_options += $common_options;
738 $backend_options += $common_backend_options;
740 $backend_options = _drush_backend_adjust_options($site_record, $command, $command_options, $backend_options);
741 $backend_options += array(
742 'drush-script' => NULL,
745 // Insure that contexts such as DRUSH_SIMULATE and NO_COLOR are included.
746 $command_options += _drush_backend_get_global_contexts($site_record);
748 // Add in command-specific options as well
749 $command_options += drush_command_get_command_specific_options($site_record, $command);
751 // If the caller has requested it, don't pull the options from the alias
752 // into the command line, but use the alias name for dispatching.
753 if (!empty($backend_options['dispatch-using-alias']) && isset($site_record['#name'])) {
754 list($post_options, $commandline_options, $drush_global_options) = _drush_backend_classify_options(array(), $command_options, $backend_options);
755 $site_record_to_dispatch = '@' . ltrim($site_record['#name'], '@');
758 list($post_options, $commandline_options, $drush_global_options) = _drush_backend_classify_options($site_record, $command_options, $backend_options);
759 $site_record_to_dispatch = '';
761 if (array_key_exists('backend-simulate', $backend_options)) {
762 $drush_global_options['simulate'] = TRUE;
764 $site_record += array('path-aliases' => array(), '#env-vars' => array());
765 $site_record['path-aliases'] += array(
766 '%drush-script' => $backend_options['drush-script'],
769 $site = (array_key_exists('#name', $site_record) && !array_key_exists($site_record['#name'], $invocation_options)) ? $site_record['#name'] : $index++;
770 $invocation_options[$site] = array(
771 'site-record' => $site_record,
772 'site-record-to-dispatch' => $site_record_to_dispatch,
773 'command' => $command,
775 'post-options' => $post_options,
776 'drush-global-options' => $drush_global_options,
777 'commandline-options' => $commandline_options,
778 'command-options' => $command_options,
779 'backend-options' => $backend_options,
783 // Calculate the length of the longest output label
784 $max_name_length = 0;
785 $label_separator = '';
786 if (!array_key_exists('no-label', $common_options) && (count($invocation_options) > 1)) {
787 $label_separator = array_key_exists('label-separator', $common_options) ? $common_options['label-separator'] : ' >> ';
788 foreach ($invocation_options as $site => $item) {
789 $backend_options = $item['backend-options'];
790 if (!array_key_exists('#output-label', $backend_options)) {
791 if (is_numeric($site)) {
792 $backend_options['#output-label'] = ' * [@self.' . $site;
793 $label_separator = '] ';
796 $backend_options['#output-label'] = $site;
798 $invocation_options[$site]['backend-options']['#output-label'] = $backend_options['#output-label'];
800 $name_len = strlen($backend_options['#output-label']);
801 if ($name_len > $max_name_length) {
802 $max_name_length = $name_len;
804 if (array_key_exists('#label-separator', $backend_options)) {
805 $label_separator = $backend_options['#label-separator'];
809 // Now pad out the output labels and add the label separator.
810 $reserve_margin = $max_name_length + strlen($label_separator);
811 foreach ($invocation_options as $site => $item) {
812 $backend_options = $item['backend-options'] + array('#output-label' => '');
813 $invocation_options[$site]['backend-options']['#output-label'] = str_pad($backend_options['#output-label'], $max_name_length, " ") . $label_separator;
814 if ($reserve_margin) {
815 $invocation_options[$site]['drush-global-options']['reserve-margin'] = $reserve_margin;
819 // Now take our prepared options and generate the command strings
821 foreach ($invocation_options as $site => $item) {
822 $site_record = $item['site-record'];
823 $site_record_to_dispatch = $item['site-record-to-dispatch'];
824 $command = $item['command'];
825 $args = $item['args'];
826 $post_options = $item['post-options'];
827 $commandline_options = $item['commandline-options'];
828 $command_options = $item['command-options'];
829 $drush_global_options = $item['drush-global-options'];
830 $backend_options = $item['backend-options'];
831 $is_remote = array_key_exists('remote-host', $site_record);
834 (isset($site_record['root']) && ($site_record['root'] != drush_get_context('DRUSH_DRUPAL_ROOT'))) ||
835 (isset($site_record['uri']) && ($site_record['uri'] != drush_get_context('DRUSH_SELECTED_URI')));
836 $os = drush_os($site_record);
837 // If the caller did not pass in a specific path to drush, then we will
838 // use a default value. For commands that are being executed on the same
839 // machine, we will use DRUSH_COMMAND, which is the path to the drush.php
840 // that is running right now. For remote commands, we will run a wrapper
841 // script instead of drush.php called drush.
842 $drush_path = $site_record['path-aliases']['%drush-script'];
843 if (!$drush_path && !$is_remote && $is_different_site) {
844 $drush_path = find_wrapper_or_launcher($site_record['root']);
846 $env_vars = $site_record['#env-vars'];
847 $php = array_key_exists('php', $site_record) ? $site_record['php'] : (array_key_exists('php', $command_options) ? $command_options['php'] : NULL);
848 $drush_command_path = drush_build_drush_command($drush_path, $php, $os, $is_remote, $env_vars);
849 $cmd = _drush_backend_generate_command($site_record, $drush_command_path . " " . _drush_backend_argument_string($drush_global_options, $os) . " " . $site_record_to_dispatch . " " . $command, $args, $commandline_options, $backend_options) . ' 2>&1';
850 $cmds[$site] = array(
852 'post-options' => $post_options,
853 'backend-options' => $backend_options,
857 return _drush_backend_invoke($cmds, $common_backend_options, $context);
861 * Find all of the drush contexts that are used to cache global values and
862 * return them in an associative array.
864 function _drush_backend_get_global_contexts($site_record) {
866 $global_option_list = drush_get_global_options(FALSE);
867 foreach ($global_option_list as $global_key => $global_metadata) {
868 if (is_array($global_metadata)) {
870 if (!array_key_exists('never-propagate', $global_metadata)) {
871 if ((array_key_exists('propagate', $global_metadata))) {
872 $value = drush_get_option($global_key);
874 elseif ((array_key_exists('propagate-cli-value', $global_metadata))) {
875 $value = drush_get_option($global_key, '', 'cli');
877 elseif ((array_key_exists('context', $global_metadata))) {
878 // If the context is declared to be a 'local-context-only',
879 // then only put it in if this is a local dispatch.
880 if (!array_key_exists('local-context-only', $global_metadata) || !array_key_exists('remote-host', $site_record)) {
881 $value = drush_get_context($global_metadata['context'], array());
884 if (!empty($value) || ($value === '0')) {
885 $result[$global_key] = $value;
894 * Take all of the values in the $command_options array, and place each of
895 * them into one of the following result arrays:
897 * - $post_options: options to be encoded as JSON and written to the
898 * standard input of the drush subprocess being executed.
899 * - $commandline_options: options to be placed on the command line of the drush
901 * - $drush_global_options: the drush global options also go on the command
902 * line, but appear before the drush command name rather than after it.
904 * Also, this function may modify $backend_options.
906 function _drush_backend_classify_options($site_record, $command_options, &$backend_options) {
907 // In 'POST' mode (the default, remove everything (except the items marked 'never-post'
908 // in the global option list) from the commandline options and put them into the post options.
909 // The post options will be json-encoded and sent to the command via stdin
910 $global_option_list = drush_get_global_options(FALSE); // These should be in the command line.
911 $additional_global_options = array();
912 if (array_key_exists('additional-global-options', $backend_options)) {
913 $additional_global_options = $backend_options['additional-global-options'];
914 $command_options += $additional_global_options;
916 $method_post = ((!array_key_exists('method', $backend_options)) || ($backend_options['method'] == 'POST'));
917 $post_options = array();
918 $commandline_options = array();
919 $drush_global_options = array();
920 $drush_local_options = array();
921 $additional_backend_options = array();
922 foreach ($site_record as $key => $value) {
923 if (!in_array($key, drush_sitealias_site_selection_keys())) {
924 if ($key[0] == '#') {
925 $backend_options[$key] = $value;
927 if (!isset($command_options[$key])) {
928 if (array_key_exists($key, $global_option_list)) {
929 $command_options[$key] = $value;
934 if (array_key_exists('drush-local-options', $backend_options)) {
935 $drush_local_options = $backend_options['drush-local-options'];
936 $command_options += $drush_local_options;
938 if (!empty($backend_options['backend']) && empty($backend_options['interactive']) && empty($backend_options['fork'])) {
939 $drush_global_options['backend'] = '2';
941 foreach ($command_options as $key => $value) {
942 $global = array_key_exists($key, $global_option_list);
946 $propagate = (!array_key_exists('never-propagate', $global_option_list[$key]));
947 $special = (array_key_exists('never-post', $global_option_list[$key]));
949 // We will allow 'merge-pathlist' contexts to be propogated. Right now
950 // these are all 'local-context-only' options; if we allowed them to
951 // propogate remotely, then we would need to get the right path separator
952 // for the remote machine.
953 if (is_array($value) && array_key_exists('merge-pathlist', $global_option_list[$key])) {
954 $value = implode(PATH_SEPARATOR, $value);
958 // Just remove options that are designated as non-propagating
959 if ($propagate === TRUE) {
960 // In METHOD POST, move command options to post options
961 if ($method_post && ($special === FALSE)) {
962 $post_options[$key] = $value;
964 // In METHOD GET, ignore options with array values
965 elseif (!is_array($value)) {
966 if ($global || array_key_exists($key, $additional_global_options)) {
967 $drush_global_options[$key] = $value;
970 $commandline_options[$key] = $value;
975 return array($post_options, $commandline_options, $drush_global_options, $additional_backend_options);
979 * Create a new pipe with proc_open, and attempt to parse the output.
981 * We use proc_open instead of exec or others because proc_open is best
982 * for doing bi-directional pipes, and we need to pass data over STDIN
983 * to the remote script.
985 * Exec also seems to exhibit some strangeness in keeping the returned
986 * data intact, in that it modifies the newline characters.
989 * The complete command line call to use.
990 * @param post_options
991 * An associative array to json-encode and pass to the remote script on stdin.
992 * @param backend_options
993 * Options for the invocation.
996 * If no commands were executed, FALSE.
998 * If one command was executed, this will return an associative array containing
999 * the data from drush_backend_output(). The result code is stored
1000 * in $result['error_status'] (0 == no error).
1002 * If multiple commands were executed, this will return an associative array
1003 * containing one item, 'concurrent', which will contain a list of the different
1004 * backend invoke results from each concurrent command.
1006 function _drush_backend_invoke($cmds, $common_backend_options = array(), $context = NULL) {
1007 if (drush_get_context('DRUSH_SIMULATE') && !array_key_exists('override-simulated', $common_backend_options) && !array_key_exists('backend-simulate', $common_backend_options)) {
1008 foreach ($cmds as $cmd) {
1009 drush_print(dt('Simulating backend invoke: !cmd', array('!cmd' => $cmd['cmd'])));
1013 foreach ($cmds as $cmd) {
1014 drush_log(dt('Backend invoke: !cmd', array('!cmd' => $cmd['cmd'])), 'command');
1016 if (!empty($common_backend_options['interactive']) || !empty($common_backend_options['fork'])) {
1017 foreach ($cmds as $cmd) {
1018 $exec_cmd = $cmd['cmd'];
1019 if (array_key_exists('fork', $common_backend_options)) {
1020 $exec_cmd .= ' --quiet &';
1023 $result_code = drush_shell_proc_open($exec_cmd);
1024 $ret = array('error_status' => $result_code);
1028 $process_limit = drush_get_option_override($common_backend_options, 'concurrency', 1);
1029 $procs = _drush_backend_proc_open($cmds, $process_limit, $context);
1030 $procs = is_array($procs) ? $procs : array($procs);
1033 foreach ($procs as $site => $proc) {
1034 if (($proc['code'] == DRUSH_APPLICATION_ERROR) && isset($common_backend_options['integrate'])) {
1035 drush_set_error('DRUSH_APPLICATION_ERROR', dt("The external command could not be executed due to an application error."));
1038 if ($proc['output']) {
1039 $values = drush_backend_parse_output($proc['output'], $proc['backend-options'], $proc['outputted']);
1040 if (is_array($values)) {
1041 $values['site'] = $site;
1045 elseif (!array_key_exists('concurrent', $ret)) {
1046 $ret = array('concurrent' => array($ret, $values));
1049 $ret['concurrent'][] = $values;
1053 $ret = drush_set_error('DRUSH_FRAMEWORK_ERROR', dt("The command could not be executed successfully (returned: !return, code: !code)", array("!return" => $proc['output'], "!code" => $proc['code'])));
1058 return empty($ret) ? FALSE : $ret;
1062 * Helper function that generates an anonymous site alias specification for
1063 * the given parameters.
1065 function drush_backend_generate_sitealias($backend_options) {
1066 // Ensure default values.
1067 $backend_options += array(
1068 'remote-host' => NULL,
1069 'remote-user' => NULL,
1070 'ssh-options' => NULL,
1071 'drush-script' => NULL,
1075 'remote-host' => $backend_options['remote-host'],
1076 'remote-user' => $backend_options['remote-user'],
1077 'ssh-options' => $backend_options['ssh-options'],
1078 '#env-vars' => $backend_options['env-vars'],
1079 'path-aliases' => array(
1080 '%drush-script' => $backend_options['drush-script'],
1086 * Generate a command to execute.
1088 * @param site_record
1089 * An array containing information used to generate the command.
1091 * Optional. A remote host to execute the drush command on.
1093 * Optional. Defaults to the current user. If you specify this, you can choose which module to send.
1095 * Optional. Defaults to "-o PasswordAuthentication=no"
1097 * Optional. An associative array of environmental variables to prefix the Drush command with.
1099 * Optional; contains paths to folders and executables useful to the command.
1101 * Optional. Defaults to the current drush.php file on the local machine, and
1102 * to simply 'drush' (the drush script in the current PATH) on remote servers.
1103 * You may also specify a different drush.php script explicitly. You will need
1104 * to set this when calling drush on a remote server if 'drush' is not in the
1105 * PATH on that machine.
1107 * A defined drush command such as 'cron', 'status' or any of the available ones such as 'drush pm'.
1109 * An array of arguments for the command.
1110 * @param command_options
1111 * Optional. An array containing options to pass to the remote script.
1112 * Array items with a numeric key are treated as optional arguments to the
1113 * command. This parameter is a reference, as any options that have been
1114 * represented as either an option, or an argument will be removed. This
1115 * allows you to pass the left over options as a JSON encoded string,
1116 * without duplicating data.
1117 * @param backend_options
1118 * Optional. An array of options for the invocation.
1119 * @see drush_backend_invoke for documentation.
1122 * A text string representing a fully escaped command.
1124 function _drush_backend_generate_command($site_record, $command, $args = array(), $command_options = array(), $backend_options = array()) {
1125 $site_record += array(
1126 'remote-host' => NULL,
1127 'remote-user' => NULL,
1128 'ssh-options' => NULL,
1129 'path-aliases' => array(),
1131 $backend_options += array(
1135 $hostname = $site_record['remote-host'];
1136 $username = $site_record['remote-user'];
1137 $ssh_options = $site_record['ssh-options'];
1138 $os = drush_os($site_record);
1140 if (drush_is_local_host($hostname)) {
1144 foreach ($command_options as $key => $arg) {
1145 if (is_numeric($key)) {
1147 unset($command_options[$key]);
1152 foreach ($args as $arg) {
1153 $cmd[] = drush_escapeshellarg($arg, $os);
1155 $option_str = _drush_backend_argument_string($command_options, $os);
1156 if (!empty($option_str)) {
1157 $cmd[] = " " . $option_str;
1159 $command = implode(' ', array_filter($cmd, 'strlen'));
1160 if (isset($hostname)) {
1161 $username = (isset($username)) ? drush_escapeshellarg($username, "LOCAL") . "@" : '';
1162 $ssh_options = $site_record['ssh-options'];
1163 $ssh_options = (isset($ssh_options)) ? $ssh_options : drush_get_option('ssh-options', "-o PasswordAuthentication=no");
1166 $ssh_cmd[] = $ssh_options;
1167 if ($backend_options['#tty']) {
1170 $ssh_cmd[] = $username . drush_escapeshellarg($hostname, "LOCAL");
1171 $ssh_cmd[] = drush_escapeshellarg($command . ' 2>&1', "LOCAL");
1173 // Remove NULLs and separate with spaces
1174 $command = implode(' ', array_filter($ssh_cmd, 'strlen'));
1181 * Map the options to a string containing all the possible arguments and options.
1184 * Optional. An array containing options to pass to the remote script.
1185 * Array items with a numeric key are treated as optional arguments to the command.
1186 * This parameter is a reference, as any options that have been represented as either an option, or an argument will be removed.
1187 * This allows you to pass the left over options as a JSON encoded string, without duplicating data.
1189 * Optional. Defaults to 'GET'.
1190 * If this parameter is set to 'POST', the $data array will be passed to the script being called as a JSON encoded string over
1191 * the STDIN pipe of that process. This is preferable if you have to pass sensitive data such as passwords and the like.
1192 * For any other value, the $data array will be collapsed down into a set of command line options to the script.
1194 * A properly formatted and escaped set of arguments and options to append to the drush.php shell command.
1196 function _drush_backend_argument_string($data, $os = NULL) {
1199 foreach ($data as $key => $value) {
1200 if (!is_array($value) && !is_object($value) && isset($value)) {
1201 if (substr($key,0,1) != '#') {
1202 $options[$key] = $value;
1208 foreach ($options as $key => $value) {
1209 $option_str .= _drush_escape_option($key, $value, $os);
1216 * Return a properly formatted and escaped command line option
1219 * The name of the option.
1221 * The value of the option.
1224 * If the value is set to TRUE, this function will return " --key"
1225 * In other cases it will return " --key='value'"
1227 function _drush_escape_option($key, $value = TRUE, $os = NULL) {
1228 if ($value !== TRUE) {
1229 $option_str = " --$key=" . drush_escapeshellarg($value, $os);
1232 $option_str = " --$key";
1238 * Read options fron STDIN during POST requests.
1240 * This function will read any text from the STDIN pipe,
1241 * and attempts to generate an associative array if valid
1242 * JSON was received.
1245 * An associative array of options, if successfull. Otherwise FALSE.
1247 function _drush_backend_get_stdin() {
1248 $fp = fopen('php://stdin', 'r');
1249 // Windows workaround: we cannot count on stream_get_contents to
1250 // return if STDIN is reading from the keyboard. We will therefore
1251 // check to see if there are already characters waiting on the
1252 // stream (as there always should be, if this is a backend call),
1253 // and if there are not, then we will exit.
1254 // This code prevents drush from hanging forever when called with
1255 // --backend from the commandline; however, overall it is still
1256 // a futile effort, as it does not seem that backend invoke can
1257 // successfully write data to that this function can read,
1258 // so the argument list and command always come out empty. :(
1259 // Perhaps stream_get_contents is the problem, and we should use
1260 // the technique described here:
1261 // http://bugs.php.net/bug.php?id=30154
1262 // n.b. the code in that issue passes '0' for the timeout in stream_select
1263 // in a loop, which is not recommended.
1264 // Note that the following DOES work:
1265 // drush ev 'print(json_encode(array("test" => "XYZZY")));' | drush status --backend
1266 // So, redirecting input is okay, it is just the proc_open that is a problem.
1267 if (drush_is_windows()) {
1268 // Note that stream_select uses reference parameters, so we need variables (can't pass a constant NULL)
1272 // Question: might we need to wait a bit for STDIN to be ready,
1273 // even if the process that called us immediately writes our parameters?
1274 // Passing '100' for the timeout here causes us to hang indefinitely
1275 // when called from the shell.
1276 $changed_streams = stream_select($read, $write, $except, 0);
1277 // Return on error (FALSE) or no changed streams (0).
1278 // Oh, according to http://php.net/manual/en/function.stream-select.php,
1279 // stream_select will return FALSE for streams returned by proc_open.
1280 // That is not applicable to us, is it? Our stream is connected to a stream
1281 // created by proc_open, but is not a stream returned by proc_open.
1282 if ($changed_streams < 1) {
1286 stream_set_blocking($fp, FALSE);
1287 $string = stream_get_contents($fp);
1289 if (trim($string)) {
1290 return json_decode($string, TRUE);