async-process/src/lib.rs

1243 lines
36 KiB
Rust
Raw Permalink Normal View History

2020-08-19 13:34:03 +00:00
//! Async interface for working with processes.
//!
//! This crate is an async version of [`std::process`].
//!
//! # Implementation
//!
//! A background thread named "async-process" is lazily created on first use, which waits for
//! spawned child processes to exit and then calls the `wait()` syscall to clean up the "zombie"
//! processes. This is unlike the `process` API in the standard library, where dropping a running
//! `Child` leaks its resources.
//!
//! This crate uses [`async-io`] for async I/O on Unix-like systems and [`blocking`] for async I/O
//! on Windows.
//!
//! [`async-io`]: https://docs.rs/async-io
//! [`blocking`]: https://docs.rs/blocking
//!
//! # Examples
//!
//! Spawn a process and collect its output:
//!
//! ```no_run
//! # futures_lite::future::block_on(async {
//! use async_process::Command;
//!
//! let out = Command::new("echo").arg("hello").arg("world").output().await?;
//! assert_eq!(out.stdout, b"hello world\n");
//! # std::io::Result::Ok(()) });
//! ```
//!
//! Read the output line-by-line as it gets produced:
//!
//! ```no_run
//! # futures_lite::future::block_on(async {
//! use async_process::{Command, Stdio};
2020-10-09 12:37:51 +00:00
//! use futures_lite::{io::BufReader, prelude::*};
2020-08-19 13:34:03 +00:00
//!
//! let mut child = Command::new("find")
//! .arg(".")
//! .stdout(Stdio::piped())
//! .spawn()?;
//!
//! let mut lines = BufReader::new(child.stdout.take().unwrap()).lines();
//!
//! while let Some(line) = lines.next().await {
//! println!("{}", line?);
//! }
//! # std::io::Result::Ok(()) });
//! ```
2020-08-18 20:19:25 +00:00
#![warn(missing_docs, missing_debug_implementations, rust_2018_idioms)]
2023-07-17 05:41:47 +00:00
#![doc(
html_favicon_url = "https://raw.githubusercontent.com/smol-rs/smol/master/assets/images/logo_fullsize_transparent.png"
)]
#![doc(
html_logo_url = "https://raw.githubusercontent.com/smol-rs/smol/master/assets/images/logo_fullsize_transparent.png"
)]
2020-08-18 20:19:25 +00:00
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
use std::convert::Infallible;
2020-08-18 20:19:25 +00:00
use std::ffi::OsStr;
2020-08-19 13:34:03 +00:00
use std::fmt;
2020-08-18 20:19:25 +00:00
use std::path::Path;
use std::pin::Pin;
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
use std::sync::atomic::{AtomicUsize, Ordering};
2020-08-18 20:19:25 +00:00
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
use std::thread;
#[cfg(unix)]
use async_io::Async;
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
use std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, OwnedFd, RawFd};
2020-08-18 20:19:25 +00:00
#[cfg(windows)]
use blocking::Unblock;
use async_lock::OnceCell;
2020-10-09 12:37:51 +00:00
use futures_lite::{future, io, prelude::*};
2020-08-18 20:19:25 +00:00
#[doc(no_inline)]
pub use std::process::{ExitStatus, Output, Stdio};
2020-08-22 16:32:49 +00:00
#[cfg(unix)]
pub mod unix;
#[cfg(windows)]
pub mod windows;
mod reaper;
mod sealed {
pub trait Sealed {}
}
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
#[cfg(test)]
static DRIVER_THREAD_SPAWNED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
/// The zombie process reaper.
///
/// This structure reaps zombie processes and emits the `SIGCHLD` signal.
struct Reaper {
/// Underlying system reaper.
sys: reaper::Reaper,
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
/// The number of tasks polling the SIGCHLD event.
///
/// If this is zero, the `async-process` thread must be spawned.
drivers: AtomicUsize,
/// Number of live `Child` instances currently running.
///
/// This is used to prevent the reaper thread from being spawned right as the program closes,
/// when the reaper thread isn't needed. This represents the number of active processes.
child_count: AtomicUsize,
}
impl Reaper {
/// Get the singleton instance of the reaper.
fn get() -> &'static Self {
static REAPER: OnceCell<Reaper> = OnceCell::new();
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
REAPER.get_or_init_blocking(|| Reaper {
sys: reaper::Reaper::new(),
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
drivers: AtomicUsize::new(0),
child_count: AtomicUsize::new(0),
})
}
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
/// Ensure that the reaper is driven.
///
/// If there are no active `driver()` callers, this will spawn the `async-process` thread.
#[inline]
fn ensure_driven(&'static self) {
if self
.drivers
.compare_exchange(0, 1, Ordering::SeqCst, Ordering::Acquire)
.is_ok()
{
self.start_driver_thread();
}
}
/// Start the `async-process` thread.
#[cold]
fn start_driver_thread(&'static self) {
#[cfg(test)]
DRIVER_THREAD_SPAWNED
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.unwrap_or_else(|_| unreachable!("Driver thread already spawned"));
thread::Builder::new()
.name("async-process".to_string())
.spawn(move || {
let driver = async move {
// No need to bump self.drivers, it was already bumped in ensure_driven.
let guard = self.sys.lock().await;
self.sys.reap(guard).await
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
};
#[cfg(unix)]
async_io::block_on(driver);
#[cfg(not(unix))]
future::block_on(driver);
})
.expect("cannot spawn async-process thread");
}
/// Register a process with this reaper.
fn register(&'static self, child: std::process::Child) -> io::Result<reaper::ChildGuard> {
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
self.ensure_driven();
self.sys.register(child)
}
}
cfg_if::cfg_if! {
if #[cfg(windows)] {
// Wraps a sync I/O type into an async I/O type.
fn wrap<T>(io: T) -> io::Result<Unblock<T>> {
Ok(Unblock::new(io))
}
} else if #[cfg(unix)] {
/// Wrap a file descriptor into a non-blocking I/O type.
fn wrap<T: std::os::unix::io::AsFd>(io: T) -> io::Result<Async<T>> {
Async::new(io)
}
}
}
/// A guard that can kill child processes, or push them into the zombie list.
2020-08-22 16:12:15 +00:00
struct ChildGuard {
inner: reaper::ChildGuard,
2020-08-22 16:12:15 +00:00
reap_on_drop: bool,
kill_on_drop: bool,
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
reaper: &'static Reaper,
2020-08-22 16:12:15 +00:00
}
impl ChildGuard {
fn get_mut(&mut self) -> &mut std::process::Child {
self.inner.get_mut()
}
}
// When the last reference to the child process is dropped, push it into the zombie list.
impl Drop for ChildGuard {
fn drop(&mut self) {
if self.kill_on_drop {
self.get_mut().kill().ok();
}
if self.reap_on_drop {
self.inner.reap(&self.reaper.sys);
}
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Decrement number of children.
self.reaper.child_count.fetch_sub(1, Ordering::Acquire);
}
}
2020-08-19 13:34:03 +00:00
/// A spawned child process.
///
/// The process can be in running or exited state. Use [`status()`][`Child::status()`] or
/// [`output()`][`Child::output()`] to wait for it to exit.
///
/// If the [`Child`] is dropped, the process keeps running in the background.
///
/// # Examples
///
/// Spawn a process and wait for it to complete:
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// Command::new("cp").arg("a.txt").arg("b.txt").status().await?;
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub struct Child {
2020-08-19 13:34:03 +00:00
/// The handle for writing to the child's standard input (stdin), if it has been captured.
2020-08-18 20:19:25 +00:00
pub stdin: Option<ChildStdin>,
2020-08-19 13:34:03 +00:00
/// The handle for reading from the child's standard output (stdout), if it has been captured.
2020-08-18 20:19:25 +00:00
pub stdout: Option<ChildStdout>,
2020-08-19 13:34:03 +00:00
/// The handle for reading from the child's standard error (stderr), if it has been captured.
2020-08-18 20:19:25 +00:00
pub stderr: Option<ChildStderr>,
/// The inner child process handle.
child: Arc<Mutex<ChildGuard>>,
2020-08-18 20:19:25 +00:00
}
impl Child {
2020-08-19 13:34:03 +00:00
/// Wraps the inner child process handle and registers it in the global process list.
///
/// The "async-process" thread waits for processes in the global list and cleans up the
/// resources when they exit.
2020-08-22 16:12:15 +00:00
fn new(cmd: &mut Command) -> io::Result<Child> {
// Make sure the reaper exists before we spawn the child process.
let reaper = Reaper::get();
2020-08-22 16:12:15 +00:00
let mut child = cmd.inner.spawn()?;
// Convert sync I/O types into async I/O types.
let stdin = child.stdin.take().map(wrap).transpose()?.map(ChildStdin);
let stdout = child.stdout.take().map(wrap).transpose()?.map(ChildStdout);
let stderr = child.stderr.take().map(wrap).transpose()?.map(ChildStderr);
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Bump the child count.
reaper.child_count.fetch_add(1, Ordering::Relaxed);
// Register the child process in the global list.
let inner = reaper.register(child)?;
2020-08-18 20:19:25 +00:00
Ok(Child {
stdin,
stdout,
stderr,
2020-08-22 16:12:15 +00:00
child: Arc::new(Mutex::new(ChildGuard {
inner,
2020-08-22 16:12:15 +00:00
reap_on_drop: cmd.reap_on_drop,
kill_on_drop: cmd.kill_on_drop,
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
reaper,
2020-08-22 16:12:15 +00:00
})),
2020-08-18 20:19:25 +00:00
})
}
2020-08-19 13:34:03 +00:00
/// Returns the OS-assigned process identifier associated with this child.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// let mut child = Command::new("ls").spawn()?;
/// println!("id: {}", child.id());
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn id(&self) -> u32 {
self.child.lock().unwrap().get_mut().id()
2020-08-18 20:19:25 +00:00
}
2020-08-19 13:34:03 +00:00
/// Forces the child process to exit.
///
/// If the child has already exited, an [`InvalidInput`] error is returned.
///
/// This is equivalent to sending a SIGKILL on Unix platforms.
///
/// [`InvalidInput`]: `std::io::ErrorKind::InvalidInput`
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// let mut child = Command::new("yes").spawn()?;
/// child.kill()?;
/// println!("exit status: {}", child.status().await?);
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn kill(&mut self) -> io::Result<()> {
self.child.lock().unwrap().get_mut().kill()
2020-08-18 20:19:25 +00:00
}
2020-08-19 13:34:03 +00:00
/// Returns the exit status if the process has exited.
///
/// Unlike [`status()`][`Child::status()`], this method will not drop the stdin handle.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// let mut child = Command::new("ls").spawn()?;
///
/// match child.try_status()? {
/// None => println!("still running"),
/// Some(status) => println!("exited with: {}", status),
/// }
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn try_status(&mut self) -> io::Result<Option<ExitStatus>> {
self.child.lock().unwrap().get_mut().try_wait()
2020-08-18 20:19:25 +00:00
}
2020-08-19 13:34:03 +00:00
/// Drops the stdin handle and waits for the process to exit.
///
/// Closing the stdin of the process helps avoid deadlocks. It ensures that the process does
/// not block waiting for input from the parent process while the parent waits for the child to
/// exit.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::{Command, Stdio};
///
/// let mut child = Command::new("cp")
/// .arg("a.txt")
/// .arg("b.txt")
/// .spawn()?;
///
/// println!("exit status: {}", child.status().await?);
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn status(&mut self) -> impl Future<Output = io::Result<ExitStatus>> {
self.stdin.take();
let child = self.child.clone();
async move { Reaper::get().sys.status(&child).await }
2020-08-18 20:19:25 +00:00
}
2020-08-19 13:34:03 +00:00
/// Drops the stdin handle and collects the output of the process.
///
/// Closing the stdin of the process helps avoid deadlocks. It ensures that the process does
/// not block waiting for input from the parent process while the parent waits for the child to
/// exit.
///
/// In order to capture the output of the process, [`Command::stdout()`] and
/// [`Command::stderr()`] must be configured with [`Stdio::piped()`].
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::{Command, Stdio};
///
/// let child = Command::new("ls")
/// .stdout(Stdio::piped())
/// .stderr(Stdio::piped())
/// .spawn()?;
///
/// let out = child.output().await?;
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn output(mut self) -> impl Future<Output = io::Result<Output>> {
2020-08-19 13:34:03 +00:00
// A future that waits for the exit status.
2020-08-18 20:19:25 +00:00
let status = self.status();
2020-08-19 13:34:03 +00:00
// A future that collects stdout.
2020-08-18 20:19:25 +00:00
let stdout = self.stdout.take();
let stdout = async move {
let mut v = Vec::new();
if let Some(mut s) = stdout {
s.read_to_end(&mut v).await?;
}
io::Result::Ok(v)
2020-08-18 20:19:25 +00:00
};
2020-08-19 13:34:03 +00:00
// A future that collects stderr.
2020-08-18 20:19:25 +00:00
let stderr = self.stderr.take();
let stderr = async move {
let mut v = Vec::new();
if let Some(mut s) = stderr {
s.read_to_end(&mut v).await?;
}
io::Result::Ok(v)
2020-08-18 20:19:25 +00:00
};
async move {
2020-08-26 21:27:37 +00:00
let (stdout, stderr) = future::try_zip(stdout, stderr).await?;
let status = status.await?;
2020-08-18 20:19:25 +00:00
Ok(Output {
status,
stdout,
stderr,
})
}
}
}
2020-08-19 13:34:03 +00:00
impl fmt::Debug for Child {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("Child")
.field("stdin", &self.stdin)
.field("stdout", &self.stdout)
.field("stderr", &self.stderr)
.finish()
}
}
/// A handle to a child process's standard input (stdin).
///
2024-04-19 01:34:15 +00:00
/// When a [`ChildStdin`] is dropped, the underlying handle gets closed. If the child process was
2020-08-19 13:34:03 +00:00
/// previously blocked on input, it becomes unblocked after dropping.
#[derive(Debug)]
2020-08-18 20:19:25 +00:00
pub struct ChildStdin(
#[cfg(windows)] Unblock<std::process::ChildStdin>,
#[cfg(unix)] Async<std::process::ChildStdin>,
);
impl ChildStdin {
/// Convert async_process::ChildStdin into std::process::Stdio.
///
/// You can use it to associate to the next process.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
/// use std::process::Stdio;
///
/// let mut ls_child = Command::new("ls").stdin(Stdio::piped()).spawn()?;
/// let stdio:Stdio = ls_child.stdin.take().unwrap().into_stdio().await?;
///
/// let mut echo_child = Command::new("echo").arg("./").stdout(stdio).spawn()?;
///
/// # std::io::Result::Ok(()) });
/// ```
pub async fn into_stdio(self) -> io::Result<std::process::Stdio> {
cfg_if::cfg_if! {
if #[cfg(windows)] {
Ok(self.0.into_inner().await.into())
} else if #[cfg(unix)] {
let child_stdin = self.0.into_inner()?;
blocking_fd(rustix::fd::AsFd::as_fd(&child_stdin))?;
Ok(child_stdin.into())
}
}
}
}
2020-08-19 13:34:03 +00:00
impl io::AsyncWrite for ChildStdin {
2020-08-18 20:19:25 +00:00
fn poll_write(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
Pin::new(&mut self.0).poll_write(cx, buf)
}
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.0).poll_flush(cx)
}
fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.0).poll_close(cx)
}
}
#[cfg(unix)]
impl AsRawFd for ChildStdin {
fn as_raw_fd(&self) -> RawFd {
self.0.as_raw_fd()
}
}
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
impl AsFd for ChildStdin {
fn as_fd(&self) -> BorrowedFd<'_> {
self.0.as_fd()
}
}
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
impl TryFrom<ChildStdin> for OwnedFd {
type Error = io::Error;
fn try_from(value: ChildStdin) -> Result<Self, Self::Error> {
value.0.try_into()
}
}
// TODO(notgull): Add mirroring AsRawHandle impls for all of the child handles
//
// at the moment this is pretty hard to do because of how they're wrapped in
// Unblock, meaning that we can't always access the underlying handle. async-fs
// gets around this by putting the handle in an Arc, but there's still some decision
// to be made about how to handle this (no pun intended)
2020-08-19 13:34:03 +00:00
/// A handle to a child process's standard output (stdout).
///
/// When a [`ChildStdout`] is dropped, the underlying handle gets closed.
#[derive(Debug)]
2020-08-18 20:19:25 +00:00
pub struct ChildStdout(
#[cfg(windows)] Unblock<std::process::ChildStdout>,
#[cfg(unix)] Async<std::process::ChildStdout>,
);
impl ChildStdout {
/// Convert async_process::ChildStdout into std::process::Stdio.
///
/// You can use it to associate to the next process.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
/// use std::process::Stdio;
/// use std::io::Read;
/// use futures_lite::AsyncReadExt;
///
/// let mut ls_child = Command::new("ls").stdout(Stdio::piped()).spawn()?;
/// let stdio:Stdio = ls_child.stdout.take().unwrap().into_stdio().await?;
///
/// let mut echo_child = Command::new("echo").stdin(stdio).stdout(Stdio::piped()).spawn()?;
/// let mut buf = vec![];
/// echo_child.stdout.take().unwrap().read(&mut buf).await;
/// # std::io::Result::Ok(()) });
/// ```
pub async fn into_stdio(self) -> io::Result<std::process::Stdio> {
cfg_if::cfg_if! {
if #[cfg(windows)] {
Ok(self.0.into_inner().await.into())
} else if #[cfg(unix)] {
let child_stdout = self.0.into_inner()?;
blocking_fd(rustix::fd::AsFd::as_fd(&child_stdout))?;
Ok(child_stdout.into())
}
}
}
}
2020-08-19 13:34:03 +00:00
impl io::AsyncRead for ChildStdout {
2020-08-18 20:19:25 +00:00
fn poll_read(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut [u8],
) -> Poll<io::Result<usize>> {
Pin::new(&mut self.0).poll_read(cx, buf)
}
}
#[cfg(unix)]
impl AsRawFd for ChildStdout {
fn as_raw_fd(&self) -> RawFd {
self.0.as_raw_fd()
}
}
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
impl AsFd for ChildStdout {
fn as_fd(&self) -> BorrowedFd<'_> {
self.0.as_fd()
}
}
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
impl TryFrom<ChildStdout> for OwnedFd {
type Error = io::Error;
fn try_from(value: ChildStdout) -> Result<Self, Self::Error> {
value.0.try_into()
}
}
2020-08-19 13:34:03 +00:00
/// A handle to a child process's standard error (stderr).
///
/// When a [`ChildStderr`] is dropped, the underlying handle gets closed.
#[derive(Debug)]
2020-08-18 20:19:25 +00:00
pub struct ChildStderr(
#[cfg(windows)] Unblock<std::process::ChildStderr>,
#[cfg(unix)] Async<std::process::ChildStderr>,
);
impl ChildStderr {
/// Convert async_process::ChildStderr into std::process::Stdio.
///
/// You can use it to associate to the next process.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
/// use std::process::Stdio;
///
/// let mut ls_child = Command::new("ls").arg("x").stderr(Stdio::piped()).spawn()?;
/// let stdio:Stdio = ls_child.stderr.take().unwrap().into_stdio().await?;
///
/// let mut echo_child = Command::new("echo").stdin(stdio).spawn()?;
/// # std::io::Result::Ok(()) });
/// ```
pub async fn into_stdio(self) -> io::Result<std::process::Stdio> {
cfg_if::cfg_if! {
if #[cfg(windows)] {
Ok(self.0.into_inner().await.into())
} else if #[cfg(unix)] {
let child_stderr = self.0.into_inner()?;
blocking_fd(rustix::fd::AsFd::as_fd(&child_stderr))?;
Ok(child_stderr.into())
}
}
}
}
2020-08-19 13:34:03 +00:00
impl io::AsyncRead for ChildStderr {
2020-08-18 20:19:25 +00:00
fn poll_read(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut [u8],
) -> Poll<io::Result<usize>> {
Pin::new(&mut self.0).poll_read(cx, buf)
}
}
#[cfg(unix)]
impl AsRawFd for ChildStderr {
fn as_raw_fd(&self) -> RawFd {
self.0.as_raw_fd()
}
}
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
impl AsFd for ChildStderr {
fn as_fd(&self) -> BorrowedFd<'_> {
self.0.as_fd()
}
}
2023-06-11 04:20:50 +00:00
#[cfg(unix)]
impl TryFrom<ChildStderr> for OwnedFd {
type Error = io::Error;
fn try_from(value: ChildStderr) -> Result<Self, Self::Error> {
value.0.try_into()
}
}
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
/// Runs the driver for the asynchronous processes.
///
/// This future takes control of global structures related to driving [`Child`]ren and reaping
/// zombie processes. These responsibilities include listening for the `SIGCHLD` signal and
/// making sure zombie processes are successfully waited on.
///
/// If multiple tasks run `driver()` at once, only one will actually drive the reaper; the other
/// ones will just sleep. If a task that is driving the reaper is dropped, a previously sleeping
/// task will take over. If all tasks driving the reaper are dropped, the "async-process" thread
/// will be spawned. The "async-process" thread just blocks on this future and will automatically
/// be spawned if no tasks are driving the reaper once a [`Child`] is created.
///
/// This future will never complete. It is intended to be ran on a background task in your
/// executor of choice.
///
/// # Examples
///
/// ```no_run
/// use async_executor::Executor;
/// use async_process::{driver, Command};
///
/// # futures_lite::future::block_on(async {
/// // Create an executor and run on it.
/// let ex = Executor::new();
/// ex.run(async {
/// // Run the driver future in the background.
/// ex.spawn(driver()).detach();
///
/// // Run a command.
/// Command::new("ls").output().await.ok();
/// }).await;
/// # });
/// ```
#[allow(clippy::manual_async_fn)]
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
#[inline]
pub fn driver() -> impl Future<Output = Infallible> + Send + 'static {
async {
// Get the reaper.
let reaper = Reaper::get();
// Make sure the reaper knows we're driving it.
reaper.drivers.fetch_add(1, Ordering::SeqCst);
// Decrement the driver count when this future is dropped.
let _guard = CallOnDrop(|| {
let prev_count = reaper.drivers.fetch_sub(1, Ordering::SeqCst);
// If this was the last driver, and there are still resources actively using the
// reaper, make sure that there is a thread driving the reaper.
if prev_count == 1
&& (reaper.child_count.load(Ordering::SeqCst) > 0 || reaper.sys.has_zombies())
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
{
reaper.ensure_driven();
}
});
// Acquire the reaper lock and start polling the SIGCHLD event.
let guard = reaper.sys.lock().await;
reaper.sys.reap(guard).await
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
}
}
2020-08-19 13:34:03 +00:00
/// A builder for spawning processes.
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// let output = if cfg!(target_os = "windows") {
/// Command::new("cmd").args(&["/C", "echo hello"]).output().await?
/// } else {
/// Command::new("sh").arg("-c").arg("echo hello").output().await?
/// };
/// # std::io::Result::Ok(()) });
/// ```
pub struct Command {
2020-08-22 16:12:15 +00:00
inner: std::process::Command,
stdin: bool,
stdout: bool,
stderr: bool,
2020-08-22 16:12:15 +00:00
reap_on_drop: bool,
kill_on_drop: bool,
2020-08-19 13:34:03 +00:00
}
2020-08-18 20:19:25 +00:00
impl Command {
2020-08-19 13:34:03 +00:00
/// Constructs a new [`Command`] for launching `program`.
///
/// The initial configuration (the working directory and environment variables) is inherited
/// from the current process.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("ls");
/// ```
2020-08-18 20:19:25 +00:00
pub fn new<S: AsRef<OsStr>>(program: S) -> Command {
Self::from(std::process::Command::new(program))
2020-08-18 20:19:25 +00:00
}
2020-08-19 13:34:03 +00:00
/// Adds a single argument to pass to the program.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("echo");
/// cmd.arg("hello");
/// cmd.arg("world");
/// ```
2020-08-18 20:19:25 +00:00
pub fn arg<S: AsRef<OsStr>>(&mut self, arg: S) -> &mut Command {
2020-08-22 16:12:15 +00:00
self.inner.arg(arg);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Adds multiple arguments to pass to the program.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("echo");
/// cmd.args(&["hello", "world"]);
/// ```
2020-08-18 20:19:25 +00:00
pub fn args<I, S>(&mut self, args: I) -> &mut Command
where
I: IntoIterator<Item = S>,
S: AsRef<OsStr>,
{
2020-08-22 16:12:15 +00:00
self.inner.args(args);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Configures an environment variable for the new process.
///
/// Note that environment variable names are case-insensitive (but case-preserving) on Windows,
/// and case-sensitive on all other platforms.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("ls");
/// cmd.env("PATH", "/bin");
/// ```
2020-08-18 20:19:25 +00:00
pub fn env<K, V>(&mut self, key: K, val: V) -> &mut Command
where
K: AsRef<OsStr>,
V: AsRef<OsStr>,
{
2020-08-22 16:12:15 +00:00
self.inner.env(key, val);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Configures multiple environment variables for the new process.
///
/// Note that environment variable names are case-insensitive (but case-preserving) on Windows,
/// and case-sensitive on all other platforms.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("ls");
/// cmd.envs(vec![("PATH", "/bin"), ("TERM", "xterm-256color")]);
/// ```
2020-08-18 20:19:25 +00:00
pub fn envs<I, K, V>(&mut self, vars: I) -> &mut Command
where
I: IntoIterator<Item = (K, V)>,
K: AsRef<OsStr>,
V: AsRef<OsStr>,
{
2020-08-22 16:12:15 +00:00
self.inner.envs(vars);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Removes an environment variable mapping.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("ls");
/// cmd.env_remove("PATH");
/// ```
2020-08-18 20:19:25 +00:00
pub fn env_remove<K: AsRef<OsStr>>(&mut self, key: K) -> &mut Command {
2020-08-22 16:12:15 +00:00
self.inner.env_remove(key);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Removes all environment variable mappings.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("ls");
/// cmd.env_clear();
/// ```
2020-08-18 20:19:25 +00:00
pub fn env_clear(&mut self) -> &mut Command {
2020-08-22 16:12:15 +00:00
self.inner.env_clear();
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Configures the working directory for the new process.
///
/// # Examples
///
/// ```
/// use async_process::Command;
///
/// let mut cmd = Command::new("ls");
/// cmd.current_dir("/");
/// ```
2020-08-18 20:19:25 +00:00
pub fn current_dir<P: AsRef<Path>>(&mut self, dir: P) -> &mut Command {
2020-08-22 16:12:15 +00:00
self.inner.current_dir(dir);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Configures the standard input (stdin) for the new process.
///
/// # Examples
///
/// ```
/// use async_process::{Command, Stdio};
///
/// let mut cmd = Command::new("cat");
/// cmd.stdin(Stdio::null());
/// ```
2020-08-18 20:19:25 +00:00
pub fn stdin<T: Into<Stdio>>(&mut self, cfg: T) -> &mut Command {
self.stdin = true;
self.inner.stdin(cfg);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Configures the standard output (stdout) for the new process.
///
/// # Examples
///
/// ```
/// use async_process::{Command, Stdio};
///
/// let mut cmd = Command::new("ls");
/// cmd.stdout(Stdio::piped());
/// ```
2020-08-18 20:19:25 +00:00
pub fn stdout<T: Into<Stdio>>(&mut self, cfg: T) -> &mut Command {
self.stdout = true;
self.inner.stdout(cfg);
2020-08-18 20:19:25 +00:00
self
}
2020-08-19 13:34:03 +00:00
/// Configures the standard error (stderr) for the new process.
///
/// # Examples
///
/// ```
/// use async_process::{Command, Stdio};
///
/// let mut cmd = Command::new("ls");
/// cmd.stderr(Stdio::piped());
/// ```
2020-08-18 20:19:25 +00:00
pub fn stderr<T: Into<Stdio>>(&mut self, cfg: T) -> &mut Command {
self.stderr = true;
self.inner.stderr(cfg);
2020-08-18 20:19:25 +00:00
self
}
2020-08-22 16:12:15 +00:00
/// Configures whether to reap the zombie process when [`Child`] is dropped.
///
/// When the process finishes, it becomes a "zombie" and some resources associated with it
/// remain until [`Child::try_status()`], [`Child::status()`], or [`Child::output()`] collects
/// its exit code.
///
/// If its exit code is never collected, the resources may leak forever. This crate has a
/// background thread named "async-process" that collects such "zombie" processes and then
/// "reaps" them, thus preventing the resource leaks.
///
/// The default value of this option is `true`.
///
/// # Examples
///
/// ```
/// use async_process::{Command, Stdio};
///
/// let mut cmd = Command::new("cat");
/// cmd.reap_on_drop(false);
/// ```
pub fn reap_on_drop(&mut self, reap_on_drop: bool) -> &mut Command {
self.reap_on_drop = reap_on_drop;
self
}
/// Configures whether to kill the process when [`Child`] is dropped.
///
/// The default value of this option is `false`.
///
/// # Examples
///
/// ```
/// use async_process::{Command, Stdio};
///
/// let mut cmd = Command::new("cat");
/// cmd.kill_on_drop(true);
/// ```
pub fn kill_on_drop(&mut self, kill_on_drop: bool) -> &mut Command {
self.kill_on_drop = kill_on_drop;
self
}
2020-08-19 13:34:03 +00:00
/// Executes the command and returns the [`Child`] handle to it.
///
/// If not configured, stdin, stdout and stderr will be set to [`Stdio::inherit()`].
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// let child = Command::new("ls").spawn()?;
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn spawn(&mut self) -> io::Result<Child> {
if !self.stdin {
self.inner.stdin(Stdio::inherit());
}
if !self.stdout {
self.inner.stdout(Stdio::inherit());
}
if !self.stderr {
self.inner.stderr(Stdio::inherit());
}
2020-08-19 13:34:03 +00:00
2020-08-22 16:12:15 +00:00
Child::new(self)
2020-08-18 20:19:25 +00:00
}
2020-08-19 13:34:03 +00:00
/// Executes the command, waits for it to exit, and returns the exit status.
///
/// If not configured, stdin, stdout and stderr will be set to [`Stdio::inherit()`].
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
/// let status = Command::new("cp")
/// .arg("a.txt")
/// .arg("b.txt")
/// .status()
/// .await?;
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn status(&mut self) -> impl Future<Output = io::Result<ExitStatus>> {
let child = self.spawn();
async { child?.status().await }
}
2020-08-19 13:34:03 +00:00
/// Executes the command and collects its output.
///
/// If not configured, stdin will be set to [`Stdio::null()`], and stdout and stderr will be
/// set to [`Stdio::piped()`].
///
/// # Examples
///
/// ```no_run
/// # futures_lite::future::block_on(async {
/// use async_process::Command;
///
2021-01-16 10:29:23 +00:00
/// let output = Command::new("cat")
/// .arg("a.txt")
/// .output()
/// .await?;
2020-08-19 13:34:03 +00:00
/// # std::io::Result::Ok(()) });
/// ```
2020-08-18 20:19:25 +00:00
pub fn output(&mut self) -> impl Future<Output = io::Result<Output>> {
if !self.stdin {
self.inner.stdin(Stdio::null());
}
if !self.stdout {
self.inner.stdout(Stdio::piped());
}
if !self.stderr {
self.inner.stderr(Stdio::piped());
}
2020-08-19 13:34:03 +00:00
2020-08-22 16:12:15 +00:00
let child = Child::new(self);
async { child?.output().await }
2020-08-18 20:19:25 +00:00
}
}
impl From<std::process::Command> for Command {
fn from(inner: std::process::Command) -> Self {
Self {
inner,
stdin: false,
stdout: false,
stderr: false,
reap_on_drop: true,
kill_on_drop: false,
}
}
}
impl fmt::Debug for Command {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
if f.alternate() {
f.debug_struct("Command")
.field("inner", &self.inner)
.field("stdin", &self.stdin)
.field("stdout", &self.stdout)
.field("stderr", &self.stderr)
.field("reap_on_drop", &self.reap_on_drop)
.field("kill_on_drop", &self.kill_on_drop)
.finish()
} else {
// Stdlib outputs command-line in Debug for Command. This does the
// same, if not in "alternate" (long pretty-printed) mode.
// This is useful for logs, for example.
fmt::Debug::fmt(&self.inner, f)
}
}
}
2021-04-24 08:45:09 +00:00
/// Moves `Fd` out of non-blocking mode.
#[cfg(unix)]
fn blocking_fd(fd: rustix::fd::BorrowedFd<'_>) -> io::Result<()> {
2022-12-30 04:39:01 +00:00
cfg_if::cfg_if! {
// ioctl(FIONBIO) sets the flag atomically, but we use this only on Linux
// for now, as with the standard library, because it seems to behave
// differently depending on the platform.
// https://github.com/rust-lang/rust/commit/efeb42be2837842d1beb47b51bb693c7474aba3d
// https://github.com/libuv/libuv/blob/e9d91fccfc3e5ff772d5da90e1c4a24061198ca0/src/unix/poll.c#L78-L80
// https://github.com/tokio-rs/mio/commit/0db49f6d5caf54b12176821363d154384357e70a
if #[cfg(target_os = "linux")] {
rustix::io::ioctl_fionbio(fd, false)?;
} else {
let previous = rustix::fs::fcntl_getfl(fd)?;
let new = previous & !rustix::fs::OFlags::NONBLOCK;
if new != previous {
rustix::fs::fcntl_setfl(fd, new)?;
}
}
}
Ok(())
}
struct CallOnDrop<F: FnMut()>(F);
impl<F: FnMut()> Drop for CallOnDrop<F> {
fn drop(&mut self) {
(self.0)();
}
}
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
#[cfg(test)]
mod test {
#[test]
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
fn polled_driver() {
use super::{driver, Command};
use futures_lite::future;
use futures_lite::prelude::*;
let is_thread_spawned =
|| super::DRIVER_THREAD_SPAWNED.load(std::sync::atomic::Ordering::SeqCst);
#[cfg(unix)]
fn command() -> Command {
let mut cmd = Command::new("sh");
cmd.arg("-c").arg("echo hello");
cmd
}
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
#[cfg(windows)]
fn command() -> Command {
let mut cmd = Command::new("cmd");
cmd.arg("/C").arg("echo hello");
cmd
}
#[cfg(unix)]
const OUTPUT: &[u8] = b"hello\n";
#[cfg(windows)]
const OUTPUT: &[u8] = b"hello\r\n";
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
future::block_on(async {
// Thread should not be spawned off the bat.
assert!(!is_thread_spawned());
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Spawn a driver.
let mut driver1 = Box::pin(driver());
future::poll_once(&mut driver1).await;
assert!(!is_thread_spawned());
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// We should be able to run the driver in parallel with a process future.
async {
(&mut driver1).await;
}
.or(async {
let output = command().output().await.unwrap();
assert_eq!(output.stdout, OUTPUT);
})
.await;
assert!(!is_thread_spawned());
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Spawn a second driver.
let mut driver2 = Box::pin(driver());
future::poll_once(&mut driver2).await;
assert!(!is_thread_spawned());
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Poll both drivers in parallel.
async {
(&mut driver1).await;
}
.or(async {
(&mut driver2).await;
})
.or(async {
let output = command().output().await.unwrap();
assert_eq!(output.stdout, OUTPUT);
})
.await;
assert!(!is_thread_spawned());
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Once one is dropped, the other should take over.
drop(driver1);
assert!(!is_thread_spawned());
feat: Add a way to run without the async-process thread I know I said that I wouldn't add any more features, but I think this is important enough. Right now, a thread called "async-process" is responsible for listening for SIGCHLD and reaping zombie processes. This listens for the SIGCHLD signal in Unix and uses a channel connected to the waitable handle on Windows. While this works, we can do better. Through async-signal, the signal was already asynchronous on Unix; we were already just using async_io::block_on to wait on the signal. After swapping out the channel used on Windows with async-channel, the process reaping function "reap" can be reimplemented as a fully asynchronous future. From here we must make sure this future is being polled at all times. To facilitate this, a function named "driver()" is added to the public API. This future acquires a lock on the reaper structure and calls the "reap()" future indefinitely. Multiple drivers can be created at once; they will just wait forever on this lock. This future is intended to be spawned onto an executor and left to run forever, making sure all child processes are signalled whenever necessary. If no tasks are running the driver future, the "async-process" thread is spawned and runs the "reap()" future itself. I've added the following controls to make sure that this system is robust: - If a "driver" task is dropped, another "driver" task will acquire the lock and keep the reaper active. - Before being dropped, the task checks to see if it is the last driver. If it is, it will spawn the "async-process" thread to be the driver. - When a Child is being created, it checks if there are any active drivers. If there are none, it spawns the "async-process" thread itself. - One concern is that the driver future wil try to spawn the "async-process" thread as the application exits and the task is being dropped, which will be unnecessary and lead to slower shutdowns. To prevent this, the future checks to see if there are any extant `Child` instances (a new refcount is added to Reaper to facilitate this). If there are none, and if there are no zombie processes, it does not spawn the additional thread. - Someone can still `mem::forget()` the driver thread. This does not lead to undefined behavior and just leads to processes being left dangling. At this point they're asking for wacky behavior. This strategy might also be viable for `async-io`, if we want to try to avoid needing to spawn the additional thread there as well. Closes #7 cc smol-rs/async-io#40 Signed-off-by: John Nunley <dev@notgull.net>
2023-10-11 00:47:46 +00:00
// Poll driver2 in parallel with a process future.
async {
(&mut driver2).await;
}
.or(async {
let output = command().output().await.unwrap();
assert_eq!(output.stdout, OUTPUT);
})
.await;
assert!(!is_thread_spawned());
// Once driver2 is dropped, the thread should not be spawned, as there are no active
// child processes..
drop(driver2);
assert!(!is_thread_spawned());
// We should now be able to poll the process future independently, it will spawn the
// thread.
let output = command().output().await.unwrap();
assert_eq!(output.stdout, OUTPUT);
assert!(is_thread_spawned());
});
}
}