mod source

module source

Marker source abstraction.

A MarkerStream is anything that can hand a stream of markers to the analysis commands, regardless of where the data physically lives (mmap’d TSV, in-memory Arrow RecordBatches, or a spilled Parquet temp file).

The existing TSV reader (crate::markers_table::MarkersTableStream) gets a blanket impl so the path-based code keeps working unchanged. The optional arrow-input feature pulls in the in-memory Arrow source and the Parquet spill source.

Modules

Re-exports

  • :rust:any:rsx_core::source::arrow_source::ArrowMarkerSource

  • :rust:any:rsx_core::source::estimator::MarkerTableSource

  • :rust:any:rsx_core::source::estimator::SizeEstimate

  • :rust:any:rsx_core::source::estimator::estimate_working_set_bytes

  • :rust:any:rsx_core::source::estimator::spill_threshold_bytes

  • :rust:any:rsx_core::source::parquet_source::ParquetMarkerSource

Traits

trait MarkerStream

Stream of markers consumed by the analysis commands.

Implementations must surface:

  • header() and groups(): same shape MarkersTableStream already exposes (header columns include id + sequence + per-individual names, groups vec has two placeholder entries followed by the popmap group for each individual column).

  • count_markers(): needed for the Bonferroni denominator.

  • for_each / par_for_each / par_fold_reduce: the iteration shapes the commands already use.

Functions

fn count_markers(&self) -> std::io::Result<u64>
fn for_each<F>(&self, f: F) -> std::io::Result<()>
where
    F: FnMut(&Marker)
fn groups(&self) -> &[String]
fn header(&self) -> &TableHeader
fn par_fold_reduce<Acc, Fold, Reduce>(&self, init: Acc, fold: Fold, reduce: Reduce) -> std::io::Result<Acc>
where
    Acc: Send + Sync + Clone,
    Fold: Fn(&mut Acc, &Marker) + Send + Sync + Clone,
    Reduce: Fn(Acc, Acc) -> Acc + Send + Sync
fn par_for_each<F>(&self, f: F) -> std::io::Result<()>
where
    F: Fn(&Marker) + Send + Sync

Implemented for

impl MarkerStream for crate::markers_table::MarkersTableStream
impl MarkerStream for ArrowMarkerSource
impl MarkerStream for MarkerTableSource
impl MarkerStream for ParquetMarkerSource