Module ashpd::desktop::input_capture

Capture input events from physical or logical devices.

Examples

A Note of Warning Regarding the GNOME Portal Implementation

xdg-desktop-portal-gnome in version 46.0 has a bug that prevents reenabling a disabled session.

Since changing barrier locations requires a session to be disabled, it is currently (as of GNOME 46) not possible to change barriers after a session has been enabled!

(the official documentation states that a InputCapture::set_pointer_barriers() request suspends the capture session but in reality the GNOME desktop portal enforces a InputCapture::disable() request in order to use InputCapture::set_pointer_barriers() )

Retrieving an Ei File Descriptor

The input capture portal is used to negotiate the input capture triggers and enable input capturing.

Actual input capture events are then communicated over a unix stream using the libei protocol.

The lifetime of an ei file descriptor is bound by a capture session.

use std::os::fd::AsRawFd;

use ashpd::desktop::input_capture::{Capabilities, InputCapture};

async fn run() -> ashpd::Result<()> {
    let input_capture = InputCapture::new().await?;
    let (session, capabilities) = input_capture
        .create_session(
            &ashpd::WindowIdentifier::default(),
            Capabilities::Keyboard | Capabilities::Pointer | Capabilities::Touchscreen,
        )
        .await?;
    eprintln!("capabilities: {capabilities}");

    let eifd = input_capture.connect_to_eis(&session).await?;
    eprintln!("eifd: {}", eifd.as_raw_fd());
    Ok(())
}

Selecting Pointer Barriers.

Input capture is triggered through pointer barriers that are provided by the client.

The provided barriers need to be positioned at the edges of outputs (monitors) and can be denied by the compositor for various reasons, such as wrong placement.

For debugging why a barrier placement failed, the logs of the active portal implementation can be useful, e.g.:

journalctl --user -xeu xdg-desktop-portal-gnome.service

The following example sets up barriers according to pos (either Left, Right, Top or Bottom).

Note that barriers positioned between two monitors will be denied and returned in the failed_barrier_ids vector.

use ashpd::desktop::input_capture::{Barrier, Capabilities, InputCapture};

#[allow(unused)]
enum Position {
    Left,
    Right,
    Top,
    Bottom,
}

async fn run() -> ashpd::Result<()> {
    let input_capture = InputCapture::new().await?;
    let (session, _capabilities) = input_capture
        .create_session(
            &ashpd::WindowIdentifier::default(),
            Capabilities::Keyboard | Capabilities::Pointer | Capabilities::Touchscreen,
        )
        .await?;

    let pos = Position::Left;
    let zones = input_capture.zones(&session).await?.response()?;
    eprintln!("zones: {zones:?}");
    let barriers = zones
        .regions()
        .iter()
        .enumerate()
        .map(|(n, r)| {
            let id = n as u32;
            let (x, y) = (r.x_offset(), r.y_offset());
            let (width, height) = (r.width() as i32, r.height() as i32);
            let barrier_pos = match pos {
                Position::Left => (x, y, x, y + height - 1), // start pos, end pos, inclusive
                Position::Right => (x + width, y, x + width, y + height - 1),
                Position::Top => (x, y, x + width - 1, y),
                Position::Bottom => (x, y + height, x + width - 1, y + height),
            };
            Barrier::new(id, barrier_pos)
        })
        .collect::<Vec<_>>();

    eprintln!("requested barriers: {barriers:?}");

    let request = input_capture
        .set_pointer_barriers(&session, &barriers, zones.zone_set())
        .await?;
    let response = request.response()?;
    let failed_barrier_ids = response.failed_barriers();

    eprintln!("failed barrier ids: {:?}", failed_barrier_ids);

    Ok(())
}

Enabling Input Capture and Retrieving Captured Input Events.

The following full example uses the reis crate for libei communication.

Input Capture can be released using ESC.

use std::{collections::HashMap, os::unix::net::UnixStream, sync::OnceLock, time::Duration};

use ashpd::desktop::input_capture::{Barrier, Capabilities, InputCapture};
use futures_util::StreamExt;
use reis::{
    ei::{self, keyboard::KeyState},
    event::{DeviceCapability, EiEvent, KeyboardKey},
    tokio::{EiConvertEventStream, EiEventStream},
};

#[allow(unused)]
enum Position {
    Left,
    Right,
    Top,
    Bottom,
}

static INTERFACES: OnceLock<HashMap<&'static str, u32>> = OnceLock::new();

async fn run() -> ashpd::Result<()> {
    let input_capture = InputCapture::new().await?;

    let (session, _cap) = input_capture
        .create_session(
            &ashpd::WindowIdentifier::default(),
            Capabilities::Keyboard | Capabilities::Pointer | Capabilities::Touchscreen,
        )
        .await?;

    // connect to eis server
    let fd = input_capture.connect_to_eis(&session).await?;

    // create unix stream from fd
    let stream = UnixStream::from(fd);
    stream.set_nonblocking(true)?;

    // create ei context
    let context = ei::Context::new(stream)?;
    context.flush().unwrap();

    let mut event_stream = EiEventStream::new(context.clone())?;
    let interfaces = INTERFACES.get_or_init(|| {
        HashMap::from([
            ("ei_connection", 1),
            ("ei_callback", 1),
            ("ei_pingpong", 1),
            ("ei_seat", 1),
            ("ei_device", 2),
            ("ei_pointer", 1),
            ("ei_pointer_absolute", 1),
            ("ei_scroll", 1),
            ("ei_button", 1),
            ("ei_keyboard", 1),
            ("ei_touchscreen", 1),
        ])
    });
    let response = reis::tokio::ei_handshake(
        &mut event_stream,
        "ashpd-mre",
        ei::handshake::ContextType::Receiver,
        interfaces,
    )
    .await
    .expect("ei handshake failed");

    let mut event_stream = EiConvertEventStream::new(event_stream, response.serial);

    let pos = Position::Left;
    let zones = input_capture.zones(&session).await?.response()?;
    eprintln!("zones: {zones:?}");
    let barriers = zones
        .regions()
        .iter()
        .enumerate()
        .map(|(n, r)| {
            let id = n as u32;
            let (x, y) = (r.x_offset(), r.y_offset());
            let (width, height) = (r.width() as i32, r.height() as i32);
            let barrier_pos = match pos {
                Position::Left => (x, y, x, y + height - 1), // start pos, end pos, inclusive
                Position::Right => (x + width, y, x + width, y + height - 1),
                Position::Top => (x, y, x + width - 1, y),
                Position::Bottom => (x, y + height, x + width - 1, y + height),
            };
            Barrier::new(id, barrier_pos)
        })
        .collect::<Vec<_>>();

    eprintln!("requested barriers: {barriers:?}");

    let request = input_capture
        .set_pointer_barriers(&session, &barriers, zones.zone_set())
        .await?;
    let response = request.response()?;
    let failed_barrier_ids = response.failed_barriers();

    eprintln!("failed barrier ids: {:?}", failed_barrier_ids);

    input_capture.enable(&session).await?;

    let mut activate_stream = input_capture.receive_activated().await?;

    loop {
        let activated = activate_stream.next().await.unwrap();

        eprintln!("activated: {activated:?}");
        loop {
            let ei_event = event_stream.next().await.unwrap().unwrap();
            eprintln!("ei event: {ei_event:?}");
            if let EiEvent::SeatAdded(seat_event) = &ei_event {
                seat_event.seat.bind_capabilities(&[
                    DeviceCapability::Pointer,
                    DeviceCapability::PointerAbsolute,
                    DeviceCapability::Keyboard,
                    DeviceCapability::Touch,
                    DeviceCapability::Scroll,
                    DeviceCapability::Button,
                ]);
                context.flush().unwrap();
            }
            if let EiEvent::DeviceAdded(_) = ei_event {
                // new device added -> restart capture
                break;
            };
            if let EiEvent::KeyboardKey(KeyboardKey { key, state, .. }) = ei_event {
                if key == 1 && state == KeyState::Press {
                    // esc pressed
                    break;
                }
            }
        }

        eprintln!("releasing input capture");
        let (x, y) = activated.cursor_position().unwrap();
        let (x, y) = (x as f64, y as f64);
        let cursor_pos = match pos {
            Position::Left => (x + 1., y),
            Position::Right => (x - 1., y),
            Position::Top => (x, y - 1.),
            Position::Bottom => (x, y + 1.),
        };
        input_capture
            .release(&session, activated.activation_id(), Some(cursor_pos))
            .await?;
    }
}

Structs

• Activated
  Indicates that an input capturing session was activated.
• Barrier
  Input Barrier.
• Deactivated
  Indicates that an input capturing session was deactivated.
• Disabled
  Indicates that an input capturing session was disabled.
• InputCapture
  Wrapper of the DBus interface: org.freedesktop.portal.InputCapture.
• Region
  A region of a Zones.
• SetPointerBarriersResponse
  A response to InputCapture::set_pointer_barriers
• Zones
  A response of InputCapture::zones.
• ZonesChanged
  Indicates that zones available to this session changed.

Enums

• Capabilities
  Supported capabilities

Type Aliases

• BarrierID
  A barrier ID.