Skip to content

feat(model): add heavy and light PDO traces - #29

Open
limonuser wants to merge 2 commits into
stack-rs:mainfrom
limonuser:feat/pdo-trace
Open

limonuser wants to merge 2 commits into
stack-rs:mainfrom
limonuser:feat/pdo-trace

Conversation

@limonuser

Copy link
Copy Markdown

Description

This is the netem-trace part of stack-rs/rattan#98 (CellReplay), following what @Lethe10137 suggested there. These are just the trace types; the cell in rattan comes after this.

Everything is behind a new pdo-model feature (also added to model):

  • HeavyPdoTrace: yields (interval since last PDO, bytes). No bandwidth anywhere.
  • LightPdoTrace: yields a LightPdo { timestamp, base_delay, opportunities }, which is one line of a CellReplay light PDO (packet train) file.
  • Models in the same style as the other ones: StaticHeavyPdo, TraceHeavyPdo, RepeatedHeavyPdoPattern, TraceLightPdo, with configs and serde (human format too).
  • load_mahimahi_heavy_pdo_trace(trace, count, pdo_bytes): lines with the same ms become one PDO with more bytes. MAHIMAHI_PDO_BYTES (1504) is for mahimahi; CellReplay's heavy files use the same format, so 1400 works for those. Repeating works like in mahimahi (the next cycle starts at the last timestamp).
  • load_cellreplay_light_pdo_trace(&str): parses time base_delay pdo0 pdo1 ... (all ms) and checks that the timestamps and PDO times don't go down.

The old load_mahimahi_trace (bw) is not changed.

Some things I wasn't sure about:

  • LightPdo only stores the PDO times, not their size. CellReplay counts every light PDO as 1400 B, so I left the size to the cell. Should it be in the trace instead?
  • I didn't add a repeat for light traces, since their timestamps are absolute and CellReplay wraps them inside the queue. I think the cell can do that, but I can add it here if you want.

Refs stack-rs/rattan#98

How Has This Been Tested

On Ubuntu 24.04 (WSL), stable Rust:

  • cargo test --features full and cargo test --features model,mahimahi,serde pass
  • cargo check --no-default-features --features pdo-model passes
  • cargo clippy --all-features --all-targets -- -Dwarnings and cargo fmt --all -- --check are clean

There's a test that replays the heavy PDOs from a mahimahi trace and checks that you get the exact same timestamps back, across 2 cycles. Plus tests for the error cases of both loaders and for serde.

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist

  • Code follows the code style of this project.
  • Changes follow the CONTRIBUTING guidelines.
  • Update necessary documentation accordingly.
  • Lint and tests pass locally with the changes.
  • Check issues and pull requests first. You don't want to duplicate effort.

Add HeavyPdoTrace ((interval since last PDO, bytes)) and LightPdoTrace
(CellReplay light PDO records: timestamp, base delay, PDO times), with
models under the new pdo-model feature, a CellReplay light PDO loader,
and load_mahimahi_heavy_pdo_trace which merges same-ms lines into one PDO.

Refs stack-rs/rattan#98

@Lethe10137 Lethe10137 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we do not need repeated version for LightPdoTrace.

However, I think that we should make the light pdo trace self-contained, both in the config (TraceLightPdoConfig), and the trace LightPdoTrace.

Consider this chain of "source of truth" of the PDO size here:

  1. The user input (e.g. in a config file for Rattan) says its 1400, either by explicitly designate the number or left that as default.
  2. Rattan loads that config file, call into_model and then it was built into an object in memory.
  3. The Cell reads the trace object, and execute that, according to the object in memory.

If 3. has to know about 1. , that's unnecessary entanglement.

Comment thread src/model/pdo.rs Outdated
pub fn build(self) -> StaticHeavyPdo {
StaticHeavyPdo {
interval: self.interval.unwrap_or_else(|| Duration::from_millis(1)),
bytes: self.bytes.unwrap_or(1500),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this a public constant named CELLREPLAY_PDO_SIZE here.

About the defualt value, considering the potential usecases:

  • loading a mahimahi trace -- We already have the 1504 bytes version constant in mahimahi.rs.
  • loading a Cellreplay trace , the defualt should be 1400

So just making this defualt as 1400.

Comment thread src/model/pdo.rs
Comment on lines +244 to +246
pub struct TraceLightPdoConfig {
pub pattern: Vec<LightPdo>,
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a PDO size field here, thus the config can be self-contained.
Use the CELLREPLAY_PDO_SIZE as default.

Comment thread src/lib.rs
/// The next_light_pdo function either returns **the next record**
/// in the sequence, or **None** if the trace goes to end.
pub trait LightPdoTrace: Send {
fn next_light_pdo(&mut self) -> Option<LightPdo>;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets add an getter, fn pdo_size(&self) -> u64

Add CELLREPLAY_PDO_SIZE (1400 bytes) and use it as the default size of
StaticHeavyPdo (was 1500). TraceLightPdoConfig gets an optional pdo_size
(defaults to CELLREPLAY_PDO_SIZE), and LightPdoTrace gets a pdo_size()
getter, so a cell can read the size from the trace itself.
@limonuser

Copy link
Copy Markdown
Author

Thanks for the review @Lethe10137! Pushed 3902c67 with all 3:

  • pub const CELLREPLAY_PDO_SIZE: u64 = 1400 in model::pdo, and StaticHeavyPdo uses it as the default now (it was 1500 before)
  • TraceLightPdoConfig has pdo_size: Option<u64>, same style as bytes in StaticHeavyPdoConfig, so it's 1400 when it's not set and it's only written to json when it is set
  • LightPdoTrace has fn pdo_size(&self) -> u64

The source of truth thing makes sense, the cell will just call trace.pdo_size() so it doesn't care if the trace came from CellReplay or somewhere else. I also re-ran the PdoReplayCell tests from rattan#216 against this branch and they still pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants