Skip to content

Add keyless Parallel search for OfficeQA rollouts - #324

Open
George Pickett (georgeatparallel) wants to merge 4 commits into
microsoft:mainfrom
georgeatparallel:parallel-search-17a2c862-6eca-4453-985f-6adc74851275
Open

George Pickett (georgeatparallel) wants to merge 4 commits into
microsoft:mainfrom
georgeatparallel:parallel-search-17a2c862-6eca-4453-985f-6adc74851275

Conversation

@georgeatparallel

@georgeatparallel George Pickett (georgeatparallel) commented Oct 18, 2026 •

Copy link
Copy Markdown

Run OfficeQA web-research rollouts with Parallel Search MCP, powered by Fast mode, without an account, API key, or access to the internal custom-search service. Parallel is opt-in: it runs only when env.search_mode=parallel is set explicitly, and auto and the default config stay offline. It needs the optional extra (pip install 'skillopt[parallel]'), so offline installs don't need mcp. Parallel receives the generated queries and the full task question, and the keyless tier is an anonymous, lower-rate free tier.

Explicit offline, Azure, and custom-service settings remain authoritative. Exec harnesses retain their tool permissions. Parallel uses different evidence from the paper's Google-backed search setting, so results are not directly comparable.

I work at Parallel.

@Yif-Yang

Copy link
Copy Markdown
Contributor

Reviewed 3b90b41860eae9e5b9bb72d8a96902a4c56da088 with the official MCP 1.27.0 wheel and an isolated dependency tree. The real SDK/HTTP-transport boundary selection passes 40 checks, including evidence consumption, explicit saved modes, exec/no-tools controls, timeouts, cancellation and error handling. No live Parallel or model call was made. These results support the value of an optional provider.

The current default change is a separate blocker: a fresh auto configuration on openai_chat or qwen_chat selects Parallel and sends the complete task question as objective, in addition to generated queries, without an explicit provider choice. Please retain the offline default and make Parallel an explicit selection with that data boundary documented. Also separate or justify the core MCP dependency and unconditional OfficeQA import so the optional capability does not unnecessarily determine offline installation behavior.

The supplied official documentation describes an anonymous lower-rate free tier, not unlimited search. Parallel evidence is not directly comparable to the paper's Google-backed setting. Your affiliation disclosure is acknowledged; this scope decision is based on default behavior and data flow, not affiliation or an unmeasured performance claim. The present head is not recommended for merge as-is.

@cursor
cursor Bot force-pushed the parallel-search-17a2c862-6eca-4453-985f-6adc74851275 branch from 3b90b41 to 83fff2a Compare October 18, 2026 17:52
@georgeatparallel

Copy link
Copy Markdown
Author

Thanks for the thorough review, Yifan Yang (@Yif-Yang). I pushed 83fff2a to address it.

The offline default is back: configs/officeqa/default.yaml and the adapter match main again, and auto now resolves to offline, so Parallel only runs with an explicit env.search_mode=parallel. mcp moved into an optional parallel extra with a lazy import, so offline installs and the OfficeQA import don't need it, and picking the mode without the extra points to pip install 'skillopt[parallel]'.

The config reference and changelog now spell out the data boundary (generated queries plus the full task question go to Parallel's hosted endpoint), describe the keyless tier as an anonymous lower-rate free tier, and keep the note that it isn't directly comparable to the paper's Google-backed setup. New tests cover each of those cases.

@Yif-Yang

Copy link
Copy Markdown
Contributor

Reviewed 83fff2aa2182dd4334ee969462c7eb16b11ba941. The current head resolves the previous default-online and mandatory-MCP blockers: the default and auto remain offline, Parallel requires explicit selection, and the built wheel imports OfficeQA without MCP installed. The documentation now states that generated queries and the full question are sent to Parallel, describes the anonymous lower-rate tier, and preserves the evidence-comparability caveat.

The current-head offline checks passed: 32 authored MCP tests and 40 independent checks covering the real MCP SDK/HTTP boundary, supported and unsupported backends, exact data flow, provider failures, discovery/search deadlines, cancellation cleanup, and the genuinely missing-extra path. Wheel packaging and strict documentation builds also passed. No live Parallel or model request was made.

The official CI run is now complete, and this PR is not merged: the Python 3.10 job fails test_whole_mcp_request_has_a_deadline at the reviewed head. The deadline does fire, but asyncio.wait_for raises asyncio.exceptions.TimeoutError on Python 3.10 while the test expects the distinct built-in TimeoutError. Python 3.11/3.12 and the other jobs pass. Please make the timeout assertion or outward exception normalization consistent across the supported Python versions, without weakening the whole-request deadline or cancellation-cleanup controls, and rerun the exact-head matrix. This is a narrow compatibility correction, not a return of the resolved default-online or mandatory-dependency findings. Local success does not override the red official job. Please also update the opening PR description: it still says fresh auto configurations select Parallel, which is no longer true. The workflow change to install .[dev,parallel] is consistent with keeping the optional provider covered.

Review baseline: 1a44050fc53a. The PR remains open; no merge was performed.

Addresses review feedback on the default selection and the core dependency.

Restore the offline default: configs/officeqa/default.yaml and
OfficeQAAdapter are byte-identical to main again, and rollout no longer
resolves an `auto` mode. `auto` (like any unrecognized value) normalizes to
`offline`, so a fresh configuration behaves exactly as it does on main.
Parallel is reached only through an explicit `env.search_mode=parallel`.

Move `mcp` out of core dependencies into a `parallel` extra and import the
Parallel client lazily, so OfficeQA imports and offline installs no longer
require it. Selecting the mode without the extra raises an error naming
`pip install 'skillopt[parallel]'`. CI installs `.[dev,parallel]` so the
provider stays covered.

Document the data boundary in docs/reference/config.md and the changelog:
selecting this mode sends the generated queries and the full task question
(as the MCP `objective`) to Parallel's hosted endpoint. Describe the keyless
tier as an anonymous, lower-rate free tier rather than unlimited search, and
keep the note that this evidence is not directly comparable to the paper's
Google-backed setting.

Tests cover that a fresh `auto` config does not select Parallel, that
explicit selection does, that the OfficeQA env imports with `mcp` absent,
and that selecting Parallel without the extra reports the install command.
MCP transport tests skip when the extra is not installed.

Co-authored-by: George Pickett <georgeatparallel@users.noreply.github.com>
The whole-request deadline fired on every supported version, but callers saw
two different exception types. `asyncio.wait_for` raises
`asyncio.exceptions.TimeoutError`, which only became an alias of the built-in
`TimeoutError` in 3.11, so the strict deadline assertion failed the CI Python
3.10 job while 3.11 and 3.12 passed.

Normalize in the provider instead of loosening the assertion: inside the
existing task-group unwrapping, re-raise an exceeded deadline as the built-in
`TimeoutError` with the original timeout error as its cause. This covers both
a direct `wait_for` timeout and one surfaced from an SDK task group, and
leaves the deadline itself and the cancellation-cleanup path untouched.
@cursor
cursor Bot force-pushed the parallel-search-17a2c862-6eca-4453-985f-6adc74851275 branch from 83fff2a to babd828 Compare October 23, 2026 17:48
@georgeatparallel

Copy link
Copy Markdown
Author

Thanks Yifan Yang (@Yif-Yang), both items are addressed in babd828, rebased onto current main (it applied cleanly).

  • Python 3.10 timeout: I reproduced the failure on 3.10. Before 3.11, asyncio.TimeoutError is a separate class from the built-in TimeoutError. The provider now converts it to the built-in TimeoutError inside the existing task-group unwrapping, so it covers a direct wait_for timeout and one that comes out of an SDK exception group. The asyncio.wait_for deadline is still around the whole request, the per-request httpx.Timeout and MCP read_timeout_seconds are unchanged, and CancelledError never reaches that block, so test_cancellation_closes_actual_mcp_session still passes. The test still expects the built-in TimeoutError. The original asyncio error is kept as __cause__.
  • PR description: it now says Parallel is opt-in through env.search_mode=parallel, that auto and the default stay offline, that it needs the optional skillopt[parallel] extra, and that Parallel receives the generated queries and the full question on the anonymous lower-rate tier.

I ran tests/test_officeqa_parallel_search.py (32 passed) and the full suite (1871 passed, 12 skipped) on Python 3.10, 3.11, and 3.12 with .[dev,parallel], and ruff is clean. The official exact-head matrix still needs to run on babd828.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants