Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 42 additions & 5 deletions crates/solana-indexer/src/indexer/decoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@
//! The decoder pulls `StreamUpdate`s from the ingester, decodes
//! settlement-program and SolFlow transactions, and persists typed events.

// TODO: `run` is unimplemented. The dispatch and persist steps that consume the
// resolved instructions below are not wired up yet.
// TODO: `decode_settlement`/`decode_solflow` and the persist path are stubbed.
// `run` drains the channel and routes each transaction's tracked instructions
// to those stubs.

use {
crate::{
Expand Down Expand Up @@ -51,10 +52,46 @@ impl Decoder {
}
}

/// Main loop. Pulls `StreamUpdate` from the receiver, runs the decode
/// pipeline, and persists typed events.
/// Main loop. Drains the channel and routes each transaction's tracked
/// instructions to their per-program decoders. Returns when the ingester
/// drops the sender.
pub async fn run(&mut self) -> Result<(), PersistenceError> {
unimplemented!()
while let Some(update) = self.rx.recv().await {
let StreamUpdate::Tx { inner, .. } = update;
self.decode(&inner);
Comment on lines +59 to +61

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to propose this should be non-blocking, considering it will bump into database IO.

Something link that:

use tokio::{sync::mpsc, task::JoinSet};

async fn run(mut rx: mpsc::Receiver<Job>) {
    let mut set = JoinSet::new(); // holds handles to independently-running tasks

    loop {
        tokio::select! {
            // pull next job, spawn it to run on its own (not driven by this loop)
            Some(job) = rx.recv() => { set.spawn(process(job)); }

            // non-blocking check for finished tasks (doesn't advance anything itself)
            Some(_) = set.join_next(), if !set.is_empty() => {}

            // channel closed AND no tasks left in flight -> done
            else => break,
        }
    }
}

(consider pseudocode, bc I haven't checked for borrow issues, etc)

@tilacog tilacog Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to note: looking at how this turned out, the Decoder is currently little more than a dispatch layer for async tasks, which made me question whether the Ingester/Decoder split is still worth keeping.

I wouldn't undo it, though. The separation of concerns still has value in principle because it gates the complexity within each component, and the channel between them is a useful abstraction on its own. That boundary could later support multiple decoder workers, or evolve into a proper bus (NATS, for example).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

decode is a stub with no IO yet, and the blocking drain is our backpressure: when decode lags, the channel fills and blocks the ingester, which an unbounded JoinSet would remove. When it's genuinely IO-bound, I'd add a few decoder workers on the shared receiver rather than spawn per message.

}
Ok(())
}

/// Route each tracked instruction in the transaction to its per-program
/// decoder.
fn decode(&self, tx: &SubscribeUpdateTransactionInfo) {
for instruction in
relevant_instructions(tx, &self.settlement_program, &self.solflow_program)
{
if instruction.program_id == self.settlement_program {
self.decode_settlement(&instruction);
} else {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this not also be gated by an if? The new additional branch could then either have a comment explaining why this can't happen, or a log warning us if we encounter an unexpected program id.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#4612 reworks this: instead of if settlement { ... } else { ... }, decode() now runs two explicit filters, program_id == settlement_program into decode_settlement and program_id == solflow_program into decode_solflow. So there's no more implicit "else means SolFlow".

self.decode_solflow(&instruction);
}
}
}

/// TODO: decode the settlement instruction data into typed events.
fn decode_settlement(&self, instruction: &ResolvedInstruction) {
tracing::debug!(
instruction_index = instruction.instruction_index,
"settlement instruction decode not implemented"
);
}

/// TODO: decode the SolFlow instruction data. The on-chain program does not
/// exist yet.
fn decode_solflow(&self, instruction: &ResolvedInstruction) {
tracing::debug!(
instruction_index = instruction.instruction_index,
"sol_flow instruction decode not implemented"
);
}
}

Expand Down
68 changes: 59 additions & 9 deletions crates/solana-indexer/src/indexer/decoder/tests.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,25 @@
use {
super::{build_account_keys, relevant_instructions},
crate::types::wire::{
CompiledInstruction,
InnerInstruction,
InnerInstructions,
Message,
SubscribeUpdateTransactionInfo,
Transaction,
TransactionStatusMeta,
super::{Decoder, build_account_keys, relevant_instructions},
crate::{
persistence::Persistence,
types::{
Signature,
channel::StreamUpdate,
slot::Slot,
wire::{
CompiledInstruction,
InnerInstruction,
InnerInstructions,
Message,
SubscribeUpdateTransactionInfo,
Transaction,
TransactionStatusMeta,
},
},
},
bytes::Bytes,
solana_sdk::pubkey::Pubkey,
tokio::sync::mpsc::Sender,
};

fn pubkey(n: u8) -> Pubkey {
Expand Down Expand Up @@ -224,3 +233,44 @@ fn corrupt_stack_height_is_clamped() {
// depth 9999 clamped to 4, so the path is bounded, not 9999 elements
assert_eq!(relevant[0].inner_ix_path, vec![0, 0, 0, 0]);
}

fn signature(n: u8) -> Signature {
Signature::from([n; 64])
}

fn test_decoder(settlement: Pubkey, solflow: Pubkey) -> (Decoder, Sender<StreamUpdate>) {
let (sender, rx) = tokio::sync::mpsc::channel(16);
let decoder = Decoder::new(Persistence {}, rx, settlement, solflow);
(decoder, sender)
}

/// A transaction carrying one settlement instruction, so draining it also
/// routes into `decode_settlement`.
fn stream_tx(slot: Slot, signature: Signature, settlement: Pubkey) -> StreamUpdate {
let info = tx_info(
vec![settlement, pubkey(8)],
vec![],
vec![],
vec![compiled(0, vec![1], vec![0])],
vec![],
);
StreamUpdate::Tx {
slot,
signature,
inner: Box::new(info),
}
}

#[tokio::test]
async fn run_drains_transactions_until_the_sender_drops() {
let (settlement, solflow) = (pubkey(1), pubkey(2));
let (mut decoder, sender) = test_decoder(settlement, solflow);

sender
.send(stream_tx(Slot(7), signature(3), settlement))
.await
.unwrap();
drop(sender);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we drop the sender before the decoder even runs it's not clear what the test is actually verifying. Should we still see the event emitted before dropping the sender?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, the test only proves the loop drains what's buffered and exits when the sender drops. It can't assert the event yet: Persistence is still a no-op stub and decode() drops its output until the store adapter lands, so there's nothing observable to check. I added a comment on the test marking that gap and where to assert the emitted event once persistence is wired (in #4612).


assert!(decoder.run().await.is_ok());
}
2 changes: 1 addition & 1 deletion crates/solana-indexer/src/types/channel.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#![expect(dead_code)]
#![allow(dead_code)]
//! Message types passed over the internal channel.
//!
//! The ingester pushes [`StreamUpdate`] into the channel to the decoder.
Expand Down
Loading