Task ID: task_e_682fdfd37d2c8323a2286f58ceec217e
Configuring language runtimes...
+ rm -- /tmp/5EZx2v-setup_script.sh
+ source ./codex_setup.sh
++ uv sync --frozen
Using CPython 3.12.3 interpreter at: /usr/bin/python3.12
Creating virtual environment at: .venv
Building scubaduck @ file:///workspace/scubaduck
Downloading ruff (11.0MiB)
Downloading pyright (5.3MiB)
Downloading duckdb (19.3MiB)
Downloading playwright (43.1MiB)
Downloading ruff
Built scubaduck @ file:///workspace/scubaduck
Downloading duckdb
Downloading playwright
Downloading pyright
Prepared 33 packages in 2.63s
Installed 33 packages in 101ms
+ blinker==1.9.0
+ certifi==2025.4.26
+ charset-normalizer==3.4.2
+ click==8.2.0
+ duckdb==1.2.2
+ execnet==2.1.1
+ flask==3.1.1
+ greenlet==3.2.2
+ idna==3.10
+ iniconfig==2.1.0
+ itsdangerous==2.2.0
+ jinja2==3.1.6
+ markupsafe==3.0.2
+ nodeenv==1.9.1
+ packaging==25.0
+ playwright==1.52.0
+ pluggy==1.6.0
+ pyee==13.0.0
+ pyright==1.1.400
+ pytest==8.3.5
+ pytest-base-url==2.1.0
+ pytest-playwright==0.7.0
+ pytest-xdist==3.6.1
+ python-dateutil==2.9.0.post0
+ python-slugify==8.0.4
+ requests==2.32.3
+ ruff==0.11.10
+ scubaduck==0.1.0 (from file:///workspace/scubaduck)
+ six==1.17.0
+ text-unidecode==1.3
+ typing-extensions==4.13.2
+ urllib3==2.4.0
+ werkzeug==3.1.3
++ source .venv/bin/activate
+++ '[' -z '' ']'
+++ '[' -n x ']'
+++ SCRIPT_PATH=.venv/bin/activate
+++ '[' .venv/bin/activate = /tmp/5EZx2v-setup_script.sh ']'
+++ deactivate nondestructive
+++ unset -f pydoc
+++ '[' -z '' ']'
+++ '[' -z '' ']'
+++ hash -r
+++ '[' -z '' ']'
+++ unset VIRTUAL_ENV
+++ unset VIRTUAL_ENV_PROMPT
+++ '[' '!' nondestructive = nondestructive ']'
+++ VIRTUAL_ENV=/workspace/scubaduck/.venv
+++ '[' linux-gnu = cygwin ']'
+++ '[' linux-gnu = msys ']'
+++ export VIRTUAL_ENV
+++ '[' -z '' ']'
+++ unset SCRIPT_PATH
+++ _OLD_VIRTUAL_PATH=/root/.cargo/bin:/root/.rbenv/shims:/root/.rbenv/bin:/root/.rbenv/shims:/root/.local/share/swiftly/bin:/root/.bun/bin:/root/.nvm/versions/node/v22.16.0/bin:/root/.pyenv/shims:3441PYENV_ROOT/shims:/root/.pyenv/bin:/usr/local/go/bin:/root/go/bin:/root/.rbenv/bin:/root/.rbenv/shims:/root/.bun/bin:/root/.local/bin:/root/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+++ PATH=/workspace/scubaduck/.venv/bin:/root/.cargo/bin:/root/.rbenv/shims:/root/.rbenv/bin:/root/.rbenv/shims:/root/.local/share/swiftly/bin:/root/.bun/bin:/root/.nvm/versions/node/v22.16.0/bin:/root/.pyenv/shims:3441PYENV_ROOT/shims:/root/.pyenv/bin:/usr/local/go/bin:/root/go/bin:/root/.rbenv/bin:/root/.rbenv/shims:/root/.bun/bin:/root/.local/bin:/root/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+++ export PATH
+++ '[' xscubaduck '!=' x ']'
+++ VIRTUAL_ENV_PROMPT=scubaduck
+++ export VIRTUAL_ENV_PROMPT
+++ '[' -z '' ']'
+++ '[' -z '' ']'
+++ _OLD_VIRTUAL_PS1=
+++ PS1='(scubaduck) '
+++ export PS1
+++ alias pydoc
+++ true
+++ hash -r
++ python -c 'import os; import duckdb; con = duckdb.connect(); con.execute(f"SET http_proxy = '\''{os.getenv("HTTP_PROXY")}'\''"); con.execute("INSTALL '\''sqlite'\'';")'
++ playwright install chromium
Downloading Chromium 136.0.7103.25 (playwright build v1169) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1169/chromium-linux.zip
167.7 MiB [] 0% 0.0s167.7 MiB [] 0% 36.8s167.7 MiB [] 0% 33.3s167.7 MiB [] 0% 22.3s167.7 MiB [] 0% 13.8s167.7 MiB [] 0% 8.8s167.7 MiB [] 1% 6.5s167.7 MiB [] 2% 5.4s167.7 MiB [] 2% 4.4s167.7 MiB [] 3% 3.5s167.7 MiB [] 4% 3.2s167.7 MiB [] 5% 3.1s167.7 MiB [] 7% 2.8s167.7 MiB [] 8% 2.6s167.7 MiB [] 9% 2.3s167.7 MiB [] 10% 2.3s167.7 MiB [] 12% 2.1s167.7 MiB [] 13% 2.0s167.7 MiB [] 14% 2.0s167.7 MiB [] 14% 1.9s167.7 MiB [] 16% 1.9s167.7 MiB [] 17% 1.8s167.7 MiB [] 18% 1.7s167.7 MiB [] 20% 1.6s167.7 MiB [] 21% 1.5s167.7 MiB [] 22% 1.5s167.7 MiB [] 24% 1.4s167.7 MiB [] 25% 1.4s167.7 MiB [] 26% 1.3s167.7 MiB [] 27% 1.3s167.7 MiB [] 28% 1.3s167.7 MiB [] 29% 1.3s167.7 MiB [] 31% 1.2s167.7 MiB [] 32% 1.2s167.7 MiB [] 33% 1.2s167.7 MiB [] 34% 1.2s167.7 MiB [] 35% 1.2s167.7 MiB [] 36% 1.2s167.7 MiB [] 37% 1.2s167.7 MiB [] 39% 1.2s167.7 MiB [] 40% 1.1s167.7 MiB [] 41% 1.1s167.7 MiB [] 42% 1.1s167.7 MiB [] 43% 1.1s167.7 MiB [] 44% 1.0s167.7 MiB [] 46% 1.0s167.7 MiB [] 47% 1.0s167.7 MiB [] 48% 1.0s167.7 MiB [] 49% 0.9s167.7 MiB [] 50% 0.9s167.7 MiB [] 52% 0.9s167.7 MiB [] 53% 0.9s167.7 MiB [] 54% 0.8s167.7 MiB [] 55% 0.8s167.7 MiB [] 57% 0.8s167.7 MiB [] 58% 0.8s167.7 MiB [] 59% 0.7s167.7 MiB [] 60% 0.7s167.7 MiB [] 62% 0.7s167.7 MiB [] 63% 0.6s167.7 MiB [] 64% 0.6s167.7 MiB [] 65% 0.6s167.7 MiB [] 66% 0.6s167.7 MiB [] 67% 0.6s167.7 MiB [] 68% 0.6s167.7 MiB [] 69% 0.5s167.7 MiB [] 70% 0.5s167.7 MiB [] 71% 0.5s167.7 MiB [] 72% 0.5s167.7 MiB [] 73% 0.5s167.7 MiB [] 74% 0.4s167.7 MiB [] 75% 0.4s167.7 MiB [] 77% 0.4s167.7 MiB [] 78% 0.4s167.7 MiB [] 79% 0.4s167.7 MiB [] 80% 0.3s167.7 MiB [] 81% 0.3s167.7 MiB [] 83% 0.3s167.7 MiB [] 84% 0.3s167.7 MiB [] 85% 0.2s167.7 MiB [] 87% 0.2s167.7 MiB [] 89% 0.2s167.7 MiB [] 90% 0.2s167.7 MiB [] 92% 0.1s167.7 MiB [] 94% 0.1s167.7 MiB [] 96% 0.1s167.7 MiB [] 97% 0.0s167.7 MiB [] 98% 0.0s167.7 MiB [] 100% 0.0s
Chromium 136.0.7103.25 (playwright build v1169) downloaded to /root/.cache/ms-playwright/chromium-1169
Downloading FFMPEG playwright build v1011 from https://cdn.playwright.dev/dbazure/download/playwright/builds/ffmpeg/1011/ffmpeg-linux.zip
2.3 MiB [] 0% 0.0s2.3 MiB [] 3% 0.5s2.3 MiB [] 7% 0.4s2.3 MiB [] 15% 0.3s2.3 MiB [] 33% 0.1s2.3 MiB [] 69% 0.0s2.3 MiB [] 100% 0.0s
FFMPEG playwright build v1011 downloaded to /root/.cache/ms-playwright/ffmpeg-1011
Downloading Chromium Headless Shell 136.0.7103.25 (playwright build v1169) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1169/chromium-headless-shell-linux.zip
101.4 MiB [] 0% 0.0s101.4 MiB [] 0% 19.6s101.4 MiB [] 0% 22.2s101.4 MiB [] 0% 10.5s101.4 MiB [] 0% 10.9s101.4 MiB [] 1% 4.5s101.4 MiB [] 3% 2.7s101.4 MiB [] 5% 2.2s101.4 MiB [] 7% 1.9s101.4 MiB [] 9% 1.6s101.4 MiB [] 11% 1.5s101.4 MiB [] 13% 1.3s101.4 MiB [] 14% 1.3s101.4 MiB [] 15% 1.4s101.4 MiB [] 16% 1.4s101.4 MiB [] 17% 1.3s101.4 MiB [] 19% 1.3s101.4 MiB [] 21% 1.2s101.4 MiB [] 23% 1.1s101.4 MiB [] 25% 1.0s101.4 MiB [] 27% 1.0s101.4 MiB [] 29% 0.9s101.4 MiB [] 30% 0.9s101.4 MiB [] 30% 1.0s101.4 MiB [] 31% 1.0s101.4 MiB [] 32% 1.0s101.4 MiB [] 34% 0.9s101.4 MiB [] 36% 0.9s101.4 MiB [] 38% 0.9s101.4 MiB [] 41% 0.8s101.4 MiB [] 42% 0.8s101.4 MiB [] 44% 0.7s101.4 MiB [] 45% 0.7s101.4 MiB [] 47% 0.7s101.4 MiB [] 49% 0.6s101.4 MiB [] 51% 0.6s101.4 MiB [] 52% 0.6s101.4 MiB [] 54% 0.6s101.4 MiB [] 55% 0.6s101.4 MiB [] 56% 0.5s101.4 MiB [] 59% 0.5s101.4 MiB [] 61% 0.5s101.4 MiB [] 63% 0.4s101.4 MiB [] 63% 0.5s101.4 MiB [] 65% 0.4s101.4 MiB [] 66% 0.4s101.4 MiB [] 68% 0.4s101.4 MiB [] 70% 0.4s101.4 MiB [] 72% 0.3s101.4 MiB [] 74% 0.3s101.4 MiB [] 76% 0.3s101.4 MiB [] 77% 0.3s101.4 MiB [] 79% 0.2s101.4 MiB [] 81% 0.2s101.4 MiB [] 83% 0.2s101.4 MiB [] 85% 0.2s101.4 MiB [] 86% 0.2s101.4 MiB [] 87% 0.1s101.4 MiB [] 88% 0.1s101.4 MiB [] 90% 0.1s101.4 MiB [] 91% 0.1s101.4 MiB [] 93% 0.1s101.4 MiB [] 94% 0.1s101.4 MiB [] 95% 0.1s101.4 MiB [] 97% 0.0s101.4 MiB [] 98% 0.0s101.4 MiB [] 99% 0.0s101.4 MiB [] 100% 0.0s
Chromium Headless Shell 136.0.7103.25 (playwright build v1169) downloaded to /root/.cache/ms-playwright/chromium_headless_shell-1169
++ echo 'source .venv/bin/activate'
I'm looking for AGENTS.md to check any relevant instructions before proceeding.
ls
bash: .venv/bin/activate: No such file or directory
root@9deabcca11c3:/workspace# ls
scubaduck
root@9deabcca11c3:/workspace# cd scubaduck
root@9deabcca11c3:/workspace/scubaduck# ls
AGENTS.md codex_setup.sh scubaduck tests
README.md pyproject.toml stubs uv.lock
root@9deabcca11c3:/workspace/scubaduck# cat AGENTS.md
- Use "source .venv/bin/activate" to activate the virtual environment before
running any commands
- ONLY use non-interactive commands like cat, sed, apply_patch to do edits.
Do NOT use interactive editors.
- Do NOT attempt to install packages. Only the packages specified in
pyproject.toml are available. You cannot add new packages. If you
desperately want another package, make a note of it in the final PR
description.
- Use conventional commits to format PR title
- There are no nested AGENTS.md files, this is the only agents file
- When using Playwright, ONLY use chromium browser. This browser is already
installed.
- Use "ruff check" to check lint, "ruff format" to autoformat files and
"pyright" to typecheck.
- When writing the PR description, include the original prompt supplied by the
user.
- When you add functionality to the server, add server tests to
tests/test_server.py. When you add features to the frontend, add frontend
tests to tests/test_web.py
I'll search the repo for 'dropdown' or 'search' to locate the relevant code.
root@9deabcca11c3:/workspace/scubaduck# ls
AGENTS.md codex_setup.sh scubaduck tests
README.md pyproject.toml stubs uv.lock
I'll search for the code related to a custom dropdown.
root@9deabcca11c3:/workspace/scubaduck# grep -R "dropdown" -n
scubaduck/static/index.html:78: .chip-dropdown { position: absolute; left: 0;
right: 0; top: 100%; background: white; border: 1px solid #ccc; max-height: 120
px; overflow-y: auto; z-index: 10; display: none; }
scubaduck/static/index.html:79: .chip-dropdown div { padding: 2px 4px; cursor
: pointer; }
scubaduck/static/index.html:80: .chip-dropdown div.highlight { background: #b
de4ff; }
scubaduck/static/index.html:82: .rel-dropdown { position: absolute; left: 0;
right: 0; top: 100%; background: white; border: 1px solid #ccc; z-index: 10; dis
play: none; }
scubaduck/static/index.html:83: .rel-dropdown div { padding: 2px 4px; cursor:
pointer; }
scubaduck/static/index.html:84: .rel-dropdown div:hover { background: #bde4ff
; }
scubaduck/static/index.html:85: .dropdown { position: relative; display: inli
ne-block; }
scubaduck/static/index.html:86: .dropdown-display {
scubaduck/static/index.html:93: .dropdown-display::after {
scubaduck/static/index.html:99: .dropdown-menu { position: absolute; left: 0;
right: 0; top: 100%; background: white; border: 1px solid #ccc; z-index: 10; ma
x-height: 160px; overflow-y: auto; display: none; }
scubaduck/static/index.html:100: .dropdown-menu input { width: 100%; box-sizi
ng: border-box; padding: 2px 4px; border: none; border-bottom: 1px solid #ccc; }
scubaduck/static/index.html:101: .dropdown-menu div { padding: 2px 4px; curso
r: pointer; }
scubaduck/static/index.html:102: .dropdown-menu div.selected { background: #b
de4ff; }
scubaduck/static/index.html:103: .dropdown-menu .option:hover { background: #
eee; }
scubaduck/static/index.html:104: .dropdown-menu input::placeholder { color: #
999; }
scubaduck/static/index.html:217: <div id="start-select" class="rel-dr
opdown" data-input="start">
scubaduck/static/index.html:235: <div id="end-select" class="rel-drop
down" data-input="end">
scubaduck/static/index.html:300: <div class="chip-dropdown"></div>
scubaduck/static/js/view_settings.js:37: // Avoid creating duplicate wrappers i
f this dropdown was already initialised.
scubaduck/static/js/view_settings.js:38: if (select.dataset.dropdownInit) {
scubaduck/static/js/view_settings.js:39: const disp = select.parentElement?.q
uerySelector('.dropdown-display');
scubaduck/static/js/view_settings.js:46: select.dataset.dropdownInit = '1';
scubaduck/static/js/view_settings.js:49: wrapper.className = 'dropdown';
scubaduck/static/js/view_settings.js:57: disp.className = 'dropdown-display';
scubaduck/static/js/view_settings.js:65: menu.className = 'dropdown-menu';
scubaduck/static/js/view_settings.js:379: const disp = tableSel.parentElement
.querySelector('.dropdown-display');
scubaduck/static/js/view_settings.js:413: document.querySelectorAll('.rel-dro
pdown').forEach(d => (d.style.display = 'none'));
scubaduck/static/js/view_settings.js:417:document.querySelectorAll('.rel-dropdow
n div').forEach(opt => {
scubaduck/static/js/view_settings.js:427: document.querySelectorAll('.rel-dropd
own').forEach(dd => {
scubaduck/static/js/view_settings.js:529: <div class="chip-dropdown"></div>
scubaduck/static/js/chip_input.js:5: const dropdown = filter.querySelector('.ch
ip-dropdown');
scubaduck/static/js/chip_input.js:11: let dropdownLocked = false;
scubaduck/static/js/chip_input.js:38: dropdown.style.display = 'none';
scubaduck/static/js/chip_input.js:39: dropdownLocked = true;
scubaduck/static/js/chip_input.js:43: if (!dropdownLocked && document.activeE
lement === input) {
scubaduck/static/js/chip_input.js:44: dropdown.style.display = 'block';
scubaduck/static/js/chip_input.js:49: Array.from(dropdown.children).forEach((
c, i) => {
scubaduck/static/js/chip_input.js:83: if (dropdown.style.display !== 'none'
) {
scubaduck/static/js/chip_input.js:84: highlight = Math.min(highlight + 1,
dropdown.children.length - 1);
scubaduck/static/js/chip_input.js:89: if (dropdown.style.display !== 'none'
) {
scubaduck/static/js/chip_input.js:100: if (dropdown.style.display !== 'none
' && dropdown.children.length > 0) {
scubaduck/static/js/chip_input.js:101: const val = dropdown.children[high
light].dataset.value;
scubaduck/static/js/chip_input.js:116: dropdown.innerHTML = '';
scubaduck/static/js/chip_input.js:135: dropdown.appendChild(d);
scubaduck/static/js/chip_input.js:147: dropdownLocked = false;
scubaduck/static/js/chip_input.js:149: dropdown.innerHTML = '';
README.md:11:We're going to build a exploratory data analysis tool like Scuba fo
r time series database (i.e., a database with a mandatory timestamp representing
the time an event occurred). We'll use DuckDB as the underlying SQL engine ser
ved from a Python server, and render the GUI/results as a webpage with vanilla H
TML and JS. We'll use choices.js to support token inputs. We define a token inp
ut to mean a text input element where as you type a dropdown displays with valid
values, and if you select one or press enter, the selection turns into a token/
chip that can only be deleted as one unit.
tests/test_web_basics.py:53:def test_time_column_dropdown(page: Any, server_url:
str) -> None:
tests/test_web_basics.py:77:def test_time_unit_dropdown(page: Any, server_url: s
tr) -> None:
tests/test_web_basics.py:92:def test_table_selector_dropdown(page: Any, server_u
rl: str) -> None:
tests/test_web_basics.py:95: disp = page.query_selector("#table + .dropdown-d
isplay")
tests/test_web_basics.py:101: assert page.query_selector("#table + .dropdown-
display + .dropdown-menu input")
tests/test_web_basics.py:206:def test_relative_dropdown(page: Any, server_url: s
tr) -> None:
tests/test_web_basics.py:216:def test_end_dropdown_now(page: Any, server_url: st
r) -> None:
tests/test_web_chips.py:6:def test_chip_dropdown_navigation(page: Any, server_ur
l: str) -> None:
tests/test_web_chips.py:18: page.wait_for_selector("#filters .filter:last-chi
ld .chip-dropdown div")
tests/test_web_chips.py:79:def test_chip_dropdown_hides_on_outside_click(page: A
ny, server_url: str) -> None:
tests/test_web_chips.py:91: page.wait_for_selector("#filters .filter:last-chi
ld .chip-dropdown div")
tests/test_web_chips.py:93: page.wait_for_selector("#filters .filter:last-chi
ld .chip-dropdown", state="hidden")
tests/test_web_chips.py:121: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:129: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_chips.py:146: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:150: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:158: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_chips.py:175: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:178: "#filters .filter:last-child .chip-dropdown
div:text('alice')"
tests/test_web_chips.py:180: page.click("#filters .filter:last-child .chip-dr
opdown div:text('alice')")
tests/test_web_chips.py:186: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_chips.py:191:def test_chip_dropdown_hides_on_column_click(page: A
ny, server_url: str) -> None:
tests/test_web_chips.py:203: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown div")
tests/test_web_chips.py:204: f.query_selector(".f-col + .dropdown-display").c
lick()
tests/test_web_chips.py:205: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown", state="hidden")
tests/test_web_chips.py:208:def test_chip_backspace_keeps_dropdown(page: Any, se
rver_url: str) -> None:
tests/test_web_chips.py:223: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown div")
tests/test_web_chips.py:233: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_table.py:196: page.wait_for_selector("#group_by_field .chip-dr
opdown div")
tests/test_web_table.py:197: options = page.locator("#group_by_field .chip-dr
opdown div").all_inner_texts()
tests/test_multi_table_web.py:39:def test_table_dropdown_persists_on_refresh(
tests/test_multi_table_web.py:51: disp = page.text_content("#table + .dropdow
n-display")
I'll open static/js/view_settings.js to check out the dropdown implementation.
grep: .venv/lib64/python3.12/site-packages/playwright/driver/node: binary file m
atches
I think the issue came from a typo in the command’s file path. Let’s open it again.
bash: .venv/bin/activate: No such file or directory
root@9deabcca11c3:/workspace# cd scubaduck
root@9deabcca11c3:/workspace/scubaduck# ls
AGENTS.md codex_setup.sh scubaduck tests
README.md pyproject.toml stubs uv.lock
I’ll check the first 100 lines of the file to get an overview.
root@9deabcca11c3:/workspace/scubaduck# sed -n '1,160p' scubaduck/static/js/view
_settings.js
// Logic for View Settings, Columns, and URL handling extracted from index.html
const allColumns = [];
const baseColumns = [];
const columnTypes = {};
const stringColumns = [];
const baseStringColumns = [];
const integerColumns = [];
const baseIntegerColumns = [];
const timeColumns = [];
const baseTimeColumns = [];
const timeColumnOptions = [];
const baseTimeColumnOptions = [];
const derivedColumns = [];
let selectedColumns = [];
let displayType = 'samples';
let groupBy = {chips: [], addChip: () => {}, renderChips: () => {}};
let defaultTimeColumn = '';
const limitInput = document.getElementById('limit');
const defaultLimit = parseInt(limitInput.value, 10);
const limitValues = {
samples: defaultLimit,
table: defaultLimit,
timeseries: 7
};
const columnValues = {
samples: [],
table: [],
timeseries: []
};
limitInput.addEventListener('input', () => {
limitValues[displayType] = parseInt(limitInput.value, 10);
limitInput.dataset.setByUser = '1';
});
function initDropdown(select) {
// Avoid creating duplicate wrappers if this dropdown was already initialised.
if (select.dataset.dropdownInit) {
const disp = select.parentElement?.querySelector('.dropdown-display');
if (disp) {
const opt = select.options[select.selectedIndex];
disp.textContent = opt ? opt.textContent : '';
}
return;
}
select.dataset.dropdownInit = '1';
const wrapper = document.createElement('div');
wrapper.className = 'dropdown';
if (select.classList.contains('f-col')) {
wrapper.classList.add('f-col');
}
select.parentNode.insertBefore(wrapper, select);
wrapper.appendChild(select);
select.style.display = 'none';
const disp = document.createElement('div');
disp.className = 'dropdown-display';
function updateDisplay() {
const opt = select.options[select.selectedIndex];
disp.textContent = opt ? opt.textContent : '';
}
updateDisplay();
wrapper.appendChild(disp);
const menu = document.createElement('div');
menu.className = 'dropdown-menu';
const search = document.createElement('input');
search.placeholder = 'Search';
menu.appendChild(search);
const list = document.createElement('div');
menu.appendChild(list);
wrapper.appendChild(menu);
function close() {
menu.style.display = 'none';
}
function open() {
renderOptions();
menu.style.display = 'block';
search.focus();
}
disp.addEventListener('click', () => {
if (menu.style.display === 'block') {
close();
} else {
open();
}
});
document.addEventListener('click', e => {
if (!wrapper.contains(e.target)) {
close();
}
});
function renderOptions() {
const q = search.value.toLowerCase();
list.innerHTML = '';
Array.from(select.options).forEach(o => {
if (!o.textContent.toLowerCase().includes(q)) return;
const div = document.createElement('div');
div.className = 'option';
if (q) {
const text = o.textContent;
const idx = text.toLowerCase().indexOf(q);
if (idx !== -1) {
div.innerHTML =
text.slice(0, idx) +
'<u>' +
text.slice(idx, idx + q.length) +
'</u>' +
text.slice(idx + q.length);
} else {
div.textContent = text;
}
} else {
div.textContent = o.textContent;
}
if (o.value === select.value) div.classList.add('selected');
div.addEventListener('mousedown', evt => {
evt.preventDefault();
select.value = o.value;
select.dispatchEvent(new Event('change'));
updateDisplay();
close();
});
list.appendChild(div);
});
}
search.addEventListener('input', renderOptions);
select.addEventListener('change', updateDisplay);
}
let orderDir = 'ASC';
const orderDirBtn = document.getElementById('order_dir');
const graphTypeSel = document.getElementById('graph_type');
function updateOrderDirButton() {
orderDirBtn.textContent = orderDir + (orderDir === 'ASC' ? ' \u25B2' : ' \u25B
C');
}
function updateDisplayTypeUI() {
const prevType = displayType;
updateSelectedColumns(prevType);
const newType = graphTypeSel.value;
const showTable = newType === 'table';
const showTS = newType === 'timeseries';
document.getElementById('group_by_field').style.display = showTable || showTS
? 'flex' : 'none';
document.getElementById('aggregate_field').style.display = showTable || showTS
? 'flex' : 'none';
document.getElementById('show_hits_field').style.display = showTable ? 'flex'
: 'none';
document.getElementById('x_axis_field').style.display = showTS ? 'flex' : 'non
e';
document.getElementById('granularity_field').style.display = showTS ? 'flex' :
'none';
document.getElementById('fill_field').style.display = showTS ? 'flex' : 'none'
;
document.querySelectorAll('#column_groups .col-group').forEach(g => {
if (g.querySelector('.col-group-header').textContent.startsWith('Strings'))
{
g.style.display = showTable || showTS ? 'none' : '';
}
});
limitValues[prevType] = parseInt(limitInput.value, 10);
if (showTS && limitValues.timeseries === undefined) {
I'm scrolling down to capture the ending parts with the dropdown code.
root@9deabcca11c3:/workspace/scubaduck# grep -n "initDropdown" -n scubaduck/stat
ic/js/view_settings.js
36:function initDropdown(select) {
353: initDropdown(orderSelect);
354: initDropdown(document.getElementById('aggregate'));
367: initDropdown(tableSel);
534: initDropdown(colSel);
I’m checking the last few lines now.
root@9deabcca11c3:/workspace/scubaduck# sed -n '160,120p' scubaduck/static/js/vi
ew_settings.js
if (showTS && limitValues.timeseries === undefined) {
Looks like the range was off—I'll try starting from 340 or 520 instead.
root@9deabcca11c3:/workspace/scubaduck# sed -n '320,400p' scubaduck/static/js/vi
ew_settings.js
ul.appendChild(li);
});
allBtn.addEventListener('click', e => {
e.preventDefault();
ul.querySelectorAll('input').forEach(cb => (cb.checked = true));
updateSelectedColumns();
});
noneBtn.addEventListener('click', e => {
e.preventDefault();
ul.querySelectorAll('input').forEach(cb => (cb.checked = false));
updateSelectedColumns();
});
div.appendChild(ul);
groupsEl.appendChild(div);
});
document.getElementById('columns_all').addEventListener('click', e => {
e.preventDefault();
groupsEl.querySelectorAll('input').forEach(cb => (cb.checked = true));
updateSelectedColumns();
});
document.getElementById('columns_none').addEventListener('click', e => {
e.preventDefault();
groupsEl.querySelectorAll('input').forEach(cb => (cb.checked = false));
updateSelectedColumns();
});
updateSelectedColumns();
columnValues.samples = allColumns.slice();
columnValues.table = [];
columnValues.timeseries = [];
groupBy = document.getElementById('group_by').closest('.field');
initChipInput(groupBy, typed =>
allColumns.filter(c => c.toLowerCase().includes(typed.toLowerCase()))
);
initDropdown(orderSelect);
initDropdown(document.getElementById('aggregate'));
});
}
let columnsInitialized = false;
fetch('/api/tables').then(r => r.json()).then(tables => {
const tableSel = document.getElementById('table');
tables.forEach(t => {
const o = document.createElement('option');
o.value = t;
o.textContent = t;
tableSel.appendChild(o);
});
initDropdown(tableSel);
const measure = document.createElement('span');
measure.style.visibility = 'hidden';
measure.style.position = 'absolute';
document.body.appendChild(measure);
let maxWidth = 0;
tables.forEach(t => {
measure.textContent = t;
const w = measure.getBoundingClientRect().width;
if (w > maxWidth) maxWidth = w;
});
measure.remove();
const disp = tableSel.parentElement.querySelector('.dropdown-display');
if (disp) disp.style.minWidth = maxWidth + 30 + 'px';
const table = parseSearch().table || tables[0];
tableSel.value = table;
tableSel.dispatchEvent(new Event('change'));
loadColumns(table).then(() => {
updateDisplayTypeUI();
addFilter();
initFromUrl();
columnsInitialized = true;
});
tableSel.addEventListener('change', () => {
loadColumns(tableSel.value).then(() => {
if (columnsInitialized) {
resetViewSettings();
applyParams({table: tableSel.value});
}
});
});
});
document.querySelectorAll('#tabs .tab').forEach(btn => {
root@9deabcca11c3:/workspace/scubaduck# grep -n "dropdown-menu" -n scubaduck/sta
tic/js/view_settings.js
65: menu.className = 'dropdown-menu';
I'll check lines 60-100 to review the open/close functions.
root@9deabcca11c3:/workspace/scubaduck# sed -n '48,110p' scubaduck/static/js/vie
w_settings.js
const wrapper = document.createElement('div');
wrapper.className = 'dropdown';
if (select.classList.contains('f-col')) {
wrapper.classList.add('f-col');
}
select.parentNode.insertBefore(wrapper, select);
wrapper.appendChild(select);
select.style.display = 'none';
const disp = document.createElement('div');
disp.className = 'dropdown-display';
function updateDisplay() {
const opt = select.options[select.selectedIndex];
disp.textContent = opt ? opt.textContent : '';
}
updateDisplay();
wrapper.appendChild(disp);
const menu = document.createElement('div');
menu.className = 'dropdown-menu';
const search = document.createElement('input');
search.placeholder = 'Search';
menu.appendChild(search);
const list = document.createElement('div');
menu.appendChild(list);
wrapper.appendChild(menu);
function close() {
menu.style.display = 'none';
}
function open() {
renderOptions();
menu.style.display = 'block';
search.focus();
}
disp.addEventListener('click', () => {
if (menu.style.display === 'block') {
close();
} else {
open();
}
});
document.addEventListener('click', e => {
if (!wrapper.contains(e.target)) {
close();
}
});
function renderOptions() {
const q = search.value.toLowerCase();
list.innerHTML = '';
Array.from(select.options).forEach(o => {
if (!o.textContent.toLowerCase().includes(q)) return;
const div = document.createElement('div');
div.className = 'option';
if (q) {
const text = o.textContent;
const idx = text.toLowerCase().indexOf(q);
if (idx !== -1) {
div.innerHTML =
text.slice(0, idx) +
'<u>' +
root@9deabcca11c3:/workspace/scubaduck# sed -n '110,190p' scubaduck/static/js/vi
ew_settings.js
'<u>' +
text.slice(idx, idx + q.length) +
'</u>' +
text.slice(idx + q.length);
} else {
div.textContent = text;
}
} else {
div.textContent = o.textContent;
}
if (o.value === select.value) div.classList.add('selected');
div.addEventListener('mousedown', evt => {
evt.preventDefault();
select.value = o.value;
select.dispatchEvent(new Event('change'));
updateDisplay();
close();
});
list.appendChild(div);
});
}
search.addEventListener('input', renderOptions);
select.addEventListener('change', updateDisplay);
}
let orderDir = 'ASC';
const orderDirBtn = document.getElementById('order_dir');
const graphTypeSel = document.getElementById('graph_type');
function updateOrderDirButton() {
orderDirBtn.textContent = orderDir + (orderDir === 'ASC' ? ' \u25B2' : ' \u25B
C');
}
function updateDisplayTypeUI() {
const prevType = displayType;
updateSelectedColumns(prevType);
const newType = graphTypeSel.value;
const showTable = newType === 'table';
const showTS = newType === 'timeseries';
document.getElementById('group_by_field').style.display = showTable || showTS
? 'flex' : 'none';
document.getElementById('aggregate_field').style.display = showTable || showTS
? 'flex' : 'none';
document.getElementById('show_hits_field').style.display = showTable ? 'flex'
: 'none';
document.getElementById('x_axis_field').style.display = showTS ? 'flex' : 'non
e';
document.getElementById('granularity_field').style.display = showTS ? 'flex' :
'none';
document.getElementById('fill_field').style.display = showTS ? 'flex' : 'none'
;
document.querySelectorAll('#column_groups .col-group').forEach(g => {
if (g.querySelector('.col-group-header').textContent.startsWith('Strings'))
{
g.style.display = showTable || showTS ? 'none' : '';
}
});
limitValues[prevType] = parseInt(limitInput.value, 10);
if (showTS && limitValues.timeseries === undefined) {
limitValues.timeseries = 7;
}
limitInput.value = limitValues[newType];
document.querySelectorAll('#column_groups input').forEach(cb => {
cb.checked = columnValues[newType].includes(cb.value);
});
if (showTS) {
document.querySelectorAll('#column_groups input').forEach(cb => {
if (isTimeColumn(cb.value) || isStringColumn(cb.value)) {
cb.checked = false;
}
});
document.getElementById('order_by').value = '';
}
updateSelectedColumns(newType);
displayType = newType;
}
function updateTimeFieldVisibility() {
const show = document.getElementById('time_column').value !== '';
document.getElementById('start').closest('.field').style.display = show
? 'flex'
: 'none';
document.getElementById('end').closest('.field').style.display = show
? 'flex'
: 'none';
document.getElementById('time_unit').style.display = show ? '' : 'none';
}
orderDirBtn.addEventListener('click', () => {
orderDir = orderDir === 'ASC' ? 'DESC' : 'ASC';
updateOrderDirButton();
I'm scrolling past line 190 to make sure I reach the file's end.
root@9deabcca11c3:/workspace/scubaduck# sed -n '190,260p' scubaduck/static/js/vi
ew_settings.js
updateOrderDirButton();
});
updateOrderDirButton();
graphTypeSel.addEventListener('change', updateDisplayTypeUI);
document.getElementById('time_column').addEventListener('change', updateTimeFiel
dVisibility);
updateTimeFieldVisibility();
function loadColumns(table) {
return fetch('/api/columns?table=' + encodeURIComponent(table)).then(r => r.js
on()).then(cols => {
const orderSelect = document.getElementById('order_by');
const xAxisSelect = document.getElementById('x_axis');
const groupsEl = document.getElementById('column_groups');
const timeColumnSelect = document.getElementById('time_column');
orderSelect.innerHTML = '';
const orderDef = document.createElement('option');
orderDef.value = '';
orderDef.textContent = '(default)';
orderSelect.appendChild(orderDef);
const samplesOpt = document.createElement('option');
samplesOpt.value = 'Samples';
samplesOpt.textContent = 'Samples';
orderSelect.appendChild(samplesOpt);
xAxisSelect.innerHTML = '';
const defOpt = document.createElement('option');
defOpt.value = '';
defOpt.textContent = '(default)';
xAxisSelect.appendChild(defOpt);
timeColumnSelect.innerHTML = '';
const noneOpt = document.createElement('option');
noneOpt.value = '';
noneOpt.textContent = '(none)';
timeColumnSelect.appendChild(noneOpt);
groupsEl.innerHTML = '';
allColumns.length = 0;
stringColumns.length = 0;
integerColumns.length = 0;
timeColumns.length = 0;
timeColumnOptions.length = 0;
baseColumns.length = 0;
baseStringColumns.length = 0;
baseIntegerColumns.length = 0;
baseTimeColumns.length = 0;
baseTimeColumnOptions.length = 0;
for (const k in columnTypes) delete columnTypes[k];
const groups = {
time: {name: 'Time', cols: [], ul: null},
integer: {name: 'Integers', cols: [], ul: null},
string: {name: 'Strings', cols: [], ul: null},
};
cols.forEach(c => {
const t = c.type.toUpperCase();
columnTypes[c.name] = c.type;
allColumns.push(c.name);
baseColumns.push(c.name);
let g = 'string';
const isNumeric = t.includes('INT') || t.includes('DECIMAL') || t.includes
('NUMERIC') || t.includes('REAL') || t.includes('DOUBLE') || t.includes('FLOAT')
|| t.includes('HUGEINT');
const isTimeType = t.includes('TIMESTAMP') || t.includes('DATE') || t.incl
udes('TIME');
if (isNumeric || isTimeType) {
timeColumnOptions.push(c.name);
baseTimeColumnOptions.push(c.name);
}
if (isTimeType) {
g = 'time';
timeColumns.push(c.name);
baseTimeColumns.push(c.name);
} else if (isNumeric) {
g = 'integer';
}
if (g === 'string') {
stringColumns.push(c.name);
baseStringColumns.push(c.name);
root@9deabcca11c3:/workspace/scubaduck# sed -n '260,340p' scubaduck/static/js/vi
ew_settings.js
baseStringColumns.push(c.name);
} else if (g === 'integer') {
integerColumns.push(c.name);
baseIntegerColumns.push(c.name);
}
groups[g].cols.push(c.name);
if (g !== 'string') {
const o = document.createElement('option');
o.value = c.name;
o.textContent = c.name;
orderSelect.appendChild(o);
}
});
timeColumns.forEach(name => {
const o = document.createElement('option');
o.value = name;
o.textContent = name;
xAxisSelect.appendChild(o);
});
timeColumnOptions.forEach(name => {
const o = document.createElement('option');
o.value = name;
o.textContent = name;
timeColumnSelect.appendChild(o);
});
xAxisSelect.value = '';
defaultTimeColumn = guessTimeColumn(cols) || '';
updateTimeFieldVisibility();
Object.keys(groups).forEach(key => {
const g = groups[key];
const div = document.createElement('div');
div.className = 'col-group';
const header = document.createElement('div');
header.className = 'col-group-header';
header.appendChild(document.createTextNode(g.name + ': '));
const links = document.createElement('span');
links.className = 'links';
const allBtn = document.createElement('a');
allBtn.href = '#';
allBtn.textContent = 'All';
const noneBtn = document.createElement('a');
noneBtn.href = '#';
noneBtn.textContent = 'None';
links.appendChild(allBtn);
links.appendChild(noneBtn);
header.appendChild(links);
div.appendChild(header);
const ul = document.createElement('ul');
g.ul = ul;
g.cols.forEach(name => {
const li = document.createElement('li');
const label = document.createElement('label');
const cb = document.createElement('input');
cb.type = 'checkbox';
cb.value = name;
cb.checked = true;
cb.addEventListener('change', updateSelectedColumns);
label.appendChild(cb);
label.appendChild(document.createTextNode(' ' + name));
li.appendChild(label);
ul.appendChild(li);
});
allBtn.addEventListener('click', e => {
e.preventDefault();
ul.querySelectorAll('input').forEach(cb => (cb.checked = true));
updateSelectedColumns();
});
noneBtn.addEventListener('click', e => {
e.preventDefault();
ul.querySelectorAll('input').forEach(cb => (cb.checked = false));
updateSelectedColumns();
});
div.appendChild(ul);
groupsEl.appendChild(div);
});
document.getElementById('columns_all').addEventListener('click', e => {
e.preventDefault();
groupsEl.querySelectorAll('input').forEach(cb => (cb.checked = true));
updateSelectedColumns();
});
document.getElementById('columns_none').addEventListener('click', e => {
The new feature involves modifying open()
to ensure the dropdown scrolls to the selected item.
root@9deabcca11c3:/workspace/scubaduck# sed -n '90,120p' scubaduck/static/index.
html
min-width: 80px;
position: relative;
}
.dropdown-display::after {
content: '\25BC';
position: absolute;
right: 4px;
pointer-events: none;
}
.dropdown-menu { position: absolute; left: 0; right: 0; top: 100%; backgroun
d: white; border: 1px solid #ccc; z-index: 10; max-height: 160px; overflow-y: au
to; display: none; }
.dropdown-menu input { width: 100%; box-sizing: border-box; padding: 2px 4px
; border: none; border-bottom: 1px solid #ccc; }
.dropdown-menu div { padding: 2px 4px; cursor: pointer; }
.dropdown-menu div.selected { background: #bde4ff; }
.dropdown-menu .option:hover { background: #eee; }
.dropdown-menu input::placeholder { color: #999; }
#filters .filter button.remove {
margin-left: 5px;
width: 20px;
flex: 0 0 auto;
padding: 0;
text-align: center;
line-height: 1;
}
#filters h4 { margin: 0 0 5px 0; }
table { border-collapse: collapse; min-width: 100%; }
th, td { border: 1px solid #ccc; padding: 4px; box-sizing: border-box; }
td.numeric { white-space: nowrap; }
td.date { white-space: nowrap; }
th { text-align: left; cursor: pointer; position: relative; }
th.sorted { color: blue; }
tr:nth-child(even) td { background: #f9f9f9; }
I need to modify the JS so the dropdown scrolls the selection near the top.
root@9deabcca11c3:/workspace/scubaduck# grep -n "dropdown" tests/test_web*.py
tests/test_web_basics.py:53:def test_time_column_dropdown(page: Any, server_url:
str) -> None:
tests/test_web_basics.py:77:def test_time_unit_dropdown(page: Any, server_url: s
tr) -> None:
tests/test_web_basics.py:92:def test_table_selector_dropdown(page: Any, server_u
rl: str) -> None:
tests/test_web_basics.py:95: disp = page.query_selector("#table + .dropdown-d
isplay")
tests/test_web_basics.py:101: assert page.query_selector("#table + .dropdown-
display + .dropdown-menu input")
tests/test_web_basics.py:206:def test_relative_dropdown(page: Any, server_url: s
tr) -> None:
tests/test_web_basics.py:216:def test_end_dropdown_now(page: Any, server_url: st
r) -> None:
tests/test_web_chips.py:6:def test_chip_dropdown_navigation(page: Any, server_ur
l: str) -> None:
tests/test_web_chips.py:18: page.wait_for_selector("#filters .filter:last-chi
ld .chip-dropdown div")
tests/test_web_chips.py:79:def test_chip_dropdown_hides_on_outside_click(page: A
ny, server_url: str) -> None:
tests/test_web_chips.py:91: page.wait_for_selector("#filters .filter:last-chi
ld .chip-dropdown div")
tests/test_web_chips.py:93: page.wait_for_selector("#filters .filter:last-chi
ld .chip-dropdown", state="hidden")
tests/test_web_chips.py:121: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:129: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_chips.py:146: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:150: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:158: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_chips.py:175: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown")
tests/test_web_chips.py:178: "#filters .filter:last-child .chip-dropdown
div:text('alice')"
tests/test_web_chips.py:180: page.click("#filters .filter:last-child .chip-dr
opdown div:text('alice')")
tests/test_web_chips.py:186: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_chips.py:191:def test_chip_dropdown_hides_on_column_click(page: A
ny, server_url: str) -> None:
tests/test_web_chips.py:203: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown div")
tests/test_web_chips.py:204: f.query_selector(".f-col + .dropdown-display").c
lick()
tests/test_web_chips.py:205: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown", state="hidden")
tests/test_web_chips.py:208:def test_chip_backspace_keeps_dropdown(page: Any, se
rver_url: str) -> None:
tests/test_web_chips.py:223: page.wait_for_selector("#filters .filter:last-ch
ild .chip-dropdown div")
tests/test_web_chips.py:233: "getComputedStyle(document.querySelector('#f
ilters .filter:last-child .chip-dropdown')).display"
tests/test_web_table.py:196: page.wait_for_selector("#group_by_field .chip-dr
opdown div")
tests/test_web_table.py:197: options = page.locator("#group_by_field .chip-dr
opdown div").all_inner_texts()
I’ll examine tests/test_web_basics.py to check how dropdowns are tested.
root@9deabcca11c3:/workspace/scubaduck# sed -n '80,120p' tests/test_web_basics.p
y
opts = page.locator("#time_unit option").all_inner_texts()
assert "ms" in opts
assert page.input_value("#time_unit") == "s"
def test_time_unit_hidden_when_no_time_column(page: Any, server_url: str) -> Non
e:
page.goto(server_url)
page.wait_for_selector("#time_column option", state="attached")
select_value(page, "#time_column", "")
assert page.is_hidden("#time_unit")
def test_table_selector_dropdown(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#table option", state="attached")
disp = page.query_selector("#table + .dropdown-display")
assert disp
assert (
page.evaluate("getComputedStyle(document.querySelector('#table')).displa
y")
== "none"
)
assert page.query_selector("#table + .dropdown-display + .dropdown-menu inpu
t")
def test_x_axis_default_entry(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#graph_type", state="attached")
select_value(page, "#graph_type", "timeseries")
page.wait_for_selector("#x_axis option", state="attached")
options = page.locator("#x_axis option").all_inner_texts()
assert "(default)" in options
assert page.input_value("#x_axis") == ""
def test_simple_filter(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.click("text=Add Filter")
filter_el = page.query_selector("#filters .filter:last-child")
assert filter_el
page.evaluate(
I'm scrolling to find the relevant tests for dropdown functionality.
root@9deabcca11c3:/workspace/scubaduck# sed -n '188,240p' tests/test_web_basics.
py
assert sidebar_overflow == "auto"
assert view_overflow == "auto"
def test_help_and_alignment(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
titles = page.evaluate(
"Array.from(document.querySelectorAll('#settings .help')).map(e => e.tit
le)"
)
assert any("start/end of the time range" in t for t in titles)
text_align = page.evaluate(
"getComputedStyle(document.querySelector('#settings label')).textAlign"
)
assert text_align == "right"
def test_relative_dropdown(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
btn = page.query_selector('[data-target="start-select"]')
assert btn
btn.click()
page.click("#start-select div:text('-3 hours')")
assert page.input_value("#start") == "-3 hours"
def test_end_dropdown_now(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.click('[data-target="end-select"]')
page.click("#end-select div:text('now')")
assert page.input_value("#end") == "now"
def test_invalid_time_error_shown(page: Any, server_url: str) -> None:
data = run_query(
page,
server_url,
start="nonsense",
end="now",
order_by="user",
)
assert "error" in data
msg = page.text_content("#view")
assert "nonsense" in msg
def test_url_query_persistence(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.wait_for_function("window.lastResults !== undefined")
I'll check if tests/web_basics has the dropdown search test. Let’s search for "dropdown."
root@9deabcca11c3:/workspace/scubaduck# grep -n "selected" -n tests/test_web_bas
ics.py
I'll open tests/test_web_chips.py to see the existing tests for this.
root@9deabcca11c3:/workspace/scubaduck# sed -n '1,80p' tests/test_web_chips.py
from __future__ import annotations
from typing import Any
def test_chip_dropdown_navigation(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.click("text=Add Filter")
f = page.query_selector("#filters .filter:last-child")
assert f
page.evaluate(
"arg => setSelectValue(arg.el.querySelector('.f-col'), arg.val)",
{"el": f, "val": "user"},
)
inp = f.query_selector(".f-val")
inp.click()
page.wait_for_selector("#filters .filter:last-child .chip-dropdown div")
page.keyboard.type("ali")
page.wait_for_selector("text=alice")
page.keyboard.press("ArrowDown")
page.keyboard.press("Enter")
chips = page.evaluate(
"Array.from(document.querySelectorAll('#filters .filter:last-child .chip
')).map(c => c.firstChild.textContent)"
)
assert chips == ["ali"]
page.click("#filters .filter:last-child .chip .x")
page.wait_for_selector(".chip", state="detached")
def test_chip_copy_and_paste(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.evaluate(
"Object.defineProperty(navigator, 'clipboard', {value:{ _data: '', write
Text(t){ this._data = t; }, readText(){ return Promise.resolve(this._data); } }}
)"
)
page.click("text=Add Filter")
f = page.query_selector("#filters .filter:last-child")
assert f
page.evaluate(
"arg => setSelectValue(arg.el.querySelector('.f-col'), arg.val)",
{"el": f, "val": "user"},
)
inp = f.query_selector(".f-val")
inp.click()
page.keyboard.type("alice")
page.keyboard.press("Enter")
inp.click()
page.keyboard.type("bob")
page.keyboard.press("Enter")
f.query_selector(".chip-copy").click()
assert page.evaluate("navigator.clipboard._data") == "alice,bob"
page.evaluate(
"var f=document.querySelector('#filters .filter:last-child'); f.chips=[]
; f.querySelectorAll('.chip').forEach(c=>c.remove())"
)
page.wait_for_selector("#filters .chip", state="detached")
inp.click()
page.evaluate(
"var dt=new DataTransfer(); dt.setData('text/plain','alice,bob'); var e=
new ClipboardEvent('paste',{clipboardData:dt}); document.querySelector('#filters
.filter:last-child .f-val').dispatchEvent(e);"
)
chips = page.evaluate(
"Array.from(document.querySelectorAll('#filters .filter:last-child .chip
')).map(c => c.firstChild.textContent)"
)
assert chips[:2] == ["alice", "bob"]
page.evaluate(
"var f=document.querySelector('#filters .filter:last-child'); f.chips=[]
; f.querySelectorAll('.chip').forEach(c=>c.remove())"
)
page.wait_for_selector("#filters .chip", state="detached")
inp.click()
page.evaluate(
"var dt=new DataTransfer(); dt.setData('text/plain','alice,bob'); var e=
new ClipboardEvent('paste',{clipboardData:dt}); Object.defineProperty(e,'shiftKe
y',{value:true}); document.querySelector('#filters .filter:last-child .f-val').d
ispatchEvent(e);"
)
chips = page.evaluate(
"Array.from(document.querySelectorAll('#filters .filter:last-child .chip
')).map(c => c.firstChild.textContent)"
)
assert chips[-1] == "alice,bob"
def test_chip_dropdown_hides_on_outside_click(page: Any, server_url: str) -> Non
e:
page.goto(server_url)
Continuing to scroll for the needed details.
root@9deabcca11c3:/workspace/scubaduck# grep -n "dropdown" -n tests/test_web_chi
ps.py
6:def test_chip_dropdown_navigation(page: Any, server_url: str) -> None:
18: page.wait_for_selector("#filters .filter:last-child .chip-dropdown div")
79:def test_chip_dropdown_hides_on_outside_click(page: Any, server_url: str) ->
None:
91: page.wait_for_selector("#filters .filter:last-child .chip-dropdown div")
93: page.wait_for_selector("#filters .filter:last-child .chip-dropdown", stat
e="hidden")
121: page.wait_for_selector("#filters .filter:last-child .chip-dropdown")
129: "getComputedStyle(document.querySelector('#filters .filter:last-chil
d .chip-dropdown')).display"
146: page.wait_for_selector("#filters .filter:last-child .chip-dropdown")
150: page.wait_for_selector("#filters .filter:last-child .chip-dropdown")
158: "getComputedStyle(document.querySelector('#filters .filter:last-chil
d .chip-dropdown')).display"
175: page.wait_for_selector("#filters .filter:last-child .chip-dropdown")
178: "#filters .filter:last-child .chip-dropdown div:text('alice')"
180: page.click("#filters .filter:last-child .chip-dropdown div:text('alice')
")
186: "getComputedStyle(document.querySelector('#filters .filter:last-chil
d .chip-dropdown')).display"
191:def test_chip_dropdown_hides_on_column_click(page: Any, server_url: str) ->
None:
203: page.wait_for_selector("#filters .filter:last-child .chip-dropdown div")
204: f.query_selector(".f-col + .dropdown-display").click()
205: page.wait_for_selector("#filters .filter:last-child .chip-dropdown", sta
te="hidden")
208:def test_chip_backspace_keeps_dropdown(page: Any, server_url: str) -> None:
223: page.wait_for_selector("#filters .filter:last-child .chip-dropdown div")
233: "getComputedStyle(document.querySelector('#filters .filter:last-chil
d .chip-dropdown')).display"
I’ll check the beginning of tests/test_web_basics.py and use it as a reference.
root@9deabcca11c3:/workspace/scubaduck# sed -n '1,40p' tests/test_web_basics.py
from __future__ import annotations
from typing import Any
from tests.web_utils import run_query, select_value
def test_range_filters(page: Any, server_url: str) -> None:
data = run_query(
page,
server_url,
start="2024-01-02 00:00:00",
end="2024-01-02 04:00:00",
order_by="user",
limit=100,
)
assert len(data["rows"]) == 2
from dateutil import parser
timestamps = [parser.parse(row[0]).replace(tzinfo=None) for row in data["row
s"]]
assert timestamps == [
parser.parse("2024-01-02 00:00:00"),
parser.parse("2024-01-02 03:00:00"),
]
def test_order_by(page: Any, server_url: str) -> None:
data = run_query(
page,
server_url,
start="2024-01-01 00:00:00",
end="2024-01-03 00:00:00",
order_by="value",
order_dir="DESC",
limit=100,
)
values = [row[2] for row in data["rows"]]
assert values == sorted(values, reverse=True)
I’m searching for the run_query function in the code. Let’s see if I can find it.
root@9deabcca11c3:/workspace/scubaduck# grep -n "run_query" -n tests/web_utils.p
y
13:def run_query(
root@9deabcca11c3:/workspace/scubaduck# sed -n '1,80p' tests/web_utils.py
from __future__ import annotations
from typing import Any
def select_value(page: Any, selector: str, value: str) -> None:
page.evaluate(
"arg => setSelectValue(arg.sel, arg.val)",
{"sel": selector, "val": value},
)
def run_query(
page: Any,
url: str,
*,
start: str | None = None,
end: str | None = None,
order_by: str | None = None,
order_dir: str | None = "ASC",
limit: int | None = None,
group_by: list[str] | None = None,
aggregate: str | None = None,
) -> dict[str, Any]:
page.goto(url)
page.wait_for_selector("#order_by option", state="attached")
page.wait_for_selector("#order_dir", state="attached")
page.wait_for_function("window.lastResults !== undefined")
if start is not None:
page.fill("#start", start)
if end is not None:
page.fill("#end", end)
if order_by is not None:
select_value(page, "#order_by", order_by)
if order_dir is not None and order_dir == "DESC":
page.click("#order_dir")
if limit is not None:
page.fill("#limit", str(limit))
if group_by is not None:
select_value(page, "#graph_type", "table")
page.evaluate(
"g => { groupBy.chips = g; groupBy.renderChips(); }",
group_by,
)
if aggregate is not None:
select_value(page, "#graph_type", "table")
select_value(page, "#aggregate", aggregate)
if page.input_value("#graph_type") != "samples":
page.click("text=Columns")
page.wait_for_selector("#column_groups input", state="attached")
if not page.is_checked("#column_groups input[value='value']"):
page.check("#column_groups input[value='value']")
order_col = order_by or page.input_value("#order_by")
if order_col and not page.is_checked(
f"#column_groups input[value='{order_col}']"
):
if page.query_selector(f"#column_groups input[value='{order_col}']")
:
page.check(f"#column_groups input[value='{order_col}']")
page.click("text=View Settings")
page.evaluate("window.lastResults = undefined")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
return page.evaluate("window.lastResults")
I'll check chip_input.js to see if it handles scrolling to the highlighted option.
root@9deabcca11c3:/workspace/scubaduck# sed -n '1,100p' scubaduck/static/js/chip
_input.js
function initChipInput(filter, fetchOptions) {
const input = filter.querySelector('.f-val');
const chipsEl = filter.querySelector('.chip-input');
const dropdown = filter.querySelector('.chip-dropdown');
const copyBtn = filter.querySelector('.chip-copy');
filter.chips = [];
filter.renderChips = renderChips;
filter.addChip = addChip;
let highlight = 0;
let dropdownLocked = false;
chipsEl.addEventListener('click', () => {
input.focus();
});
function renderChips() {
chipsEl.querySelectorAll('.chip').forEach(c => c.remove());
filter.chips.forEach((v, i) => {
const span = document.createElement('span');
span.className = 'chip';
span.textContent = v;
const x = document.createElement('span');
x.className = 'x';
x.textContent = '✖';
x.addEventListener('click', e => {
e.stopPropagation();
filter.chips.splice(i, 1);
renderChips();
input.focus();
});
span.appendChild(x);
chipsEl.insertBefore(span, input);
});
}
function hideDropdown() {
dropdown.style.display = 'none';
dropdownLocked = true;
}
function showDropdown() {
if (!dropdownLocked && document.activeElement === input) {
dropdown.style.display = 'block';
}
}
function updateHighlight() {
Array.from(dropdown.children).forEach((c, i) => {
c.classList.toggle('highlight', i === highlight);
});
}
function addChip(val) {
if (!val) return;
const i = filter.chips.indexOf(val);
if (i !== -1) {
filter.chips.splice(i, 1);
} else {
filter.chips.push(val);
}
input.value = '';
renderChips();
}
copyBtn.addEventListener('click', () => {
navigator.clipboard && navigator.clipboard.writeText(filter.chips.join(','))
;
});
input.addEventListener('paste', e => {
e.preventDefault();
const text = e.clipboardData.getData('text');
if (e.shiftKey) {
addChip(text.trim());
} else {
text.split(',').forEach(t => addChip(t.trim()));
}
hideDropdown();
});
input.addEventListener('keydown', e => {
if (e.key === 'ArrowDown') {
if (dropdown.style.display !== 'none') {
highlight = Math.min(highlight + 1, dropdown.children.length - 1);
updateHighlight();
}
e.preventDefault();
} else if (e.key === 'ArrowUp') {
if (dropdown.style.display !== 'none') {
highlight = Math.max(highlight - 1, 0);
updateHighlight();
}
e.preventDefault();
} else if (e.key === 'Backspace' && input.value === '') {
if (filter.chips.length > 0) {
filter.chips.pop();
renderChips();
}
} else if (e.key === 'Enter') {
if (dropdown.style.display !== 'none' && dropdown.children.length > 0) {
The file uses highlight and arrow keys. I'll open it and scroll more.
root@9deabcca11c3:/workspace/scubaduck# sed -n '100,200p' scubaduck/static/js/ch
ip_input.js
if (dropdown.style.display !== 'none' && dropdown.children.length > 0) {
const val = dropdown.children[highlight].dataset.value;
if (val !== input.value.trim()) {
addChip(val);
} else {
addChip(input.value.trim());
}
} else {
addChip(input.value.trim());
}
hideDropdown();
e.preventDefault();
}
});
function renderDropdown(vals) {
dropdown.innerHTML = '';
const typed = input.value.trim();
if (typed) {
vals.splice(1, 0, typed);
}
vals.forEach((v, i) => {
const d = document.createElement('div');
d.textContent = v;
d.dataset.value = v;
d.addEventListener('mouseover', () => {
highlight = i;
updateHighlight();
});
d.addEventListener('mousedown', evt => {
evt.preventDefault();
addChip(v);
hideDropdown();
input.blur();
});
dropdown.appendChild(d);
});
if (vals.length) {
highlight = 0;
updateHighlight();
showDropdown();
} else {
hideDropdown();
}
}
function loadOptions() {
dropdownLocked = false;
if (!fetchOptions) {
dropdown.innerHTML = '';
return;
}
Promise.resolve(fetchOptions(input.value, filter)).then(values => {
renderDropdown(Array.isArray(values) ? values : []);
});
}
input.addEventListener('focus', loadOptions);
input.addEventListener('input', loadOptions);
document.addEventListener('click', evt => {
if (evt.target !== input) {
hideDropdown();
}
});
}
The dropdown doesn't currently scroll to the selected item. I'll need to add scroll logic in open()
. The layout in HTML is a menu
containing a search
input and a list
div. I'll adjust the scrollTop
of the menu, considering the offset height of the search input. Time to check the markup and confirm.
root@9deabcca11c3:/workspace/scubaduck# grep -n "dropdown-menu" -n scubaduck/sta
tic/index.html
99: .dropdown-menu { position: absolute; left: 0; right: 0; top: 100%; backgr
ound: white; border: 1px solid #ccc; z-index: 10; max-height: 160px; overflow-y:
auto; display: none; }
100: .dropdown-menu input { width: 100%; box-sizing: border-box; padding: 2px
4px; border: none; border-bottom: 1px solid #ccc; }
101: .dropdown-menu div { padding: 2px 4px; cursor: pointer; }
102: .dropdown-menu div.selected { background: #bde4ff; }
103: .dropdown-menu .option:hover { background: #eee; }
104: .dropdown-menu input::placeholder { color: #999; }
I need the selected item close to the top, but with a small gap, maybe 4 px.
Set menu.scrollTop = selected.offsetTop - search.offsetHeight - 4;
for proper positioning.
root@9deabcca11c3:/workspace/scubaduck# grep -n "scrollTop" -R scubaduck
I'll check tests/test_web_basics.py for any usage of 'scrollTop' or 'scroll'.
root@9deabcca11c3:/workspace/scubaduck# grep -n "scroll" -n tests/test_web_basic
s.py
I couldn't find a test for scroll, so I'll add a new one. Let me search for existing tests.
root@9deabcca11c3:/workspace/scubaduck# grep -n "scroll" -R tests
tests/test_web_table.py:126: "var v=document.getElementById('view'); v.sc
rollWidth > v.clientWidth"
tests/test_web_table.py:352: "var v=document.getElementById('view'); v.sc
rollWidth > v.clientWidth"
tests/test_web_timeseries.py:292: "var v=document.getElementById('view');
v.scrollWidth > v.clientWidth"
I’ll add a test for dropdown scroll, checking style in test_web_basics.py.
"arg => setSelectValue(arg.el.querySelector('.f-col'), arg.val)",
{"el": filter_el, "val": "user"},
)
val_input = filter_el.query_selector(".f-val")
val_input.click()
page.keyboard.type("alice")
page.keyboard.press("Enter")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
data = page.evaluate("window.lastResults")
assert len(data["rows"]) == 2
assert all(row[3] == "alice" for row in data["rows"])
def test_default_filter_and_layout(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
count = page.evaluate("document.querySelectorAll('#filters .filter').length"
)
assert count == 1
last_is_button = page.evaluate(
"document.querySelector('#filters').lastElementChild.id === 'add_filter'
"
)
assert last_is_button
position = page.evaluate(
"getComputedStyle(document.querySelector('#filters .filter button.remove
')).position"
)
assert position == "static"
def test_filter_remove_alignment(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
diff = page.evaluate(
"() => { const r=document.querySelector('#filters .filter-row').getBound
ingClientRect(); const x=document.querySelector('#filters .filter-row button.rem
ove').getBoundingClientRect(); return Math.abs(r.right - x.right); }"
)
assert diff <= 1
def test_header_and_tabs(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
header = page.text_content("#header")
assert "sample.csv" in header
assert "events" in header
assert page.is_visible("#settings")
assert page.is_hidden("#columns")
page.click("text=Columns")
assert page.is_visible("#columns")
cols = [c.strip() for c in page.locator("#column_groups li").all_inner_texts
()]
assert "timestamp" in cols
assert "event" in cols
page.click("text=View Settings")
assert page.is_visible("#settings")
btn_color = page.evaluate(
"getComputedStyle(document.querySelector('#dive')).backgroundColor"
)
assert "rgb(0, 128, 0)" == btn_color
sidebar_overflow = page.evaluate(
"getComputedStyle(document.querySelector('#sidebar')).overflowY"
)
view_overflow = page.evaluate(
"getComputedStyle(document.querySelector('#view')).overflowY"
)
assert sidebar_overflow == "auto"
I'm looking at how open dropdowns affect scroll. I'll check style around file's end.
root@9deabcca11c3:/workspace/scubaduck# sed -n '240,320p' tests/test_web_basics.
py
page.wait_for_function("window.lastResults !== undefined")
page.fill("#start", "2024-01-01 00:00:00")
page.fill("#end", "2024-01-02 00:00:00")
page.fill("#limit", "1")
page.evaluate("window.lastResults = undefined")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
first_url = page.url
first_rows = page.evaluate("window.lastResults.rows.length")
page.fill("#limit", "2")
page.evaluate("window.lastResults = undefined")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
second_url = page.url
second_rows = page.evaluate("window.lastResults.rows.length")
assert second_rows != first_rows
assert first_url != second_url
page.go_back()
page.wait_for_function("window.lastResults !== undefined")
assert page.url == first_url
assert page.evaluate("window.lastResults.rows.length") == first_rows
def test_load_from_url(page: Any, server_url: str) -> None:
url = (
f"{server_url}?start=2024-01-01%2000:00:00&end=2024-01-02%2000:00:00"
"&order_by=timestamp&limit=2"
)
page.goto(url)
page.wait_for_selector("#order_by option", state="attached")
page.wait_for_function("window.lastResults !== undefined")
assert page.input_value("#start") == "2024-01-01 00:00:00"
assert page.input_value("#end") == "2024-01-02 00:00:00"
assert page.input_value("#limit") == "2"
assert page.evaluate("window.lastResults.rows.length") == 2
I’ll look for test files related to multi_table or timeseries to add my test.
root@9deabcca11c3:/workspace/scubaduck# ls tests
conftest.py test_server_time.py test_web_graph.py
test_multi_table_web.py test_server_timeseries.py test_web_table.py
test_server.py test_web.py test_web_timeseries.py
test_server_basic.py test_web_basics.py web_utils.py
test_server_db_types.py test_web_chips.py
test_server_errors.py test_web_columns.py
I'll check tests/test_web.py for style and consistency with other features.
from typing import Any
from tests.web_utils import select_value
def test_order_by_implicit_column(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.click("text=Columns")
page.wait_for_selector("#column_groups input", state="attached")
page.uncheck("#column_groups input[value='timestamp']")
page.click("text=View Settings")
page.fill("#start", "2024-01-01 00:00:00")
page.fill("#end", "2024-01-02 00:00:00")
select_value(page, "#order_by", "timestamp")
page.fill("#limit", "10")
page.evaluate("window.lastResults = undefined")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
data = page.evaluate("window.lastResults")
headers = page.locator("#results th").all_inner_texts()
assert "timestamp" in headers
assert len(data["rows"][0]) == 4
page.click("text=Columns")
page.wait_for_selector("#column_groups input", state="attached")
assert not page.is_checked("#column_groups input[value='timestamp']")
I'm considering different dropdowns to test the scrollTop behavior based on available options.
ic/index.html
216: <button type="button" class="rel-btn" data-target="start-select"
>▼</button>
217: <div id="start-select" class="rel-dropdown" data-input="start">
root@9deabcca11c3:/workspace/scubaduck# sed -n '210,240p' scubaduck/static/index
.html
</select>
</div>
<div class="field">
<label>Start<span class="help" title="Sets the start/end of the time r
ange to query. Can be any kind of datetime string. For example: 'April 23, 2014'
or 'yesterday'.">[?]</span></label>
<div class="rel-box">
<input id="start" type="text" />
<button type="button" class="rel-btn" data-target="start-select">	
660;</button>
<div id="start-select" class="rel-dropdown" data-input="start">
<div data-value="-1 hour">-1 hour</div>
<div data-value="-3 hours">-3 hours</div>
<div data-value="-12 hours">-12 hours</div>
<div data-value="-1 day">-1 day</div>
<div data-value="-3 days">-3 days</div>
<div data-value="-1 week">-1 week</div>
<div data-value="-1 fortnight">-1 fortnight</div>
<div data-value="-30 days">-30 days</div>
<div data-value="-90 days">-90 days</div>
</div>
</div>
</div>
<div class="field">
<label>End<span class="help" title="Sets the start/end of the time ran
ge to query. Can be any kind of datetime string. For example: 'April 23, 2014' o
r 'yesterday'.">[?]</span></label>
<div class="rel-box">
<input id="end" type="text" />
<button type="button" class="rel-btn" data-target="end-select">φ
0;</button>
<div id="end-select" class="rel-dropdown" data-input="end">
<div data-value="now">now</div>
<div data-value="-1 hour">-1 hour</div>
<div data-value="-3 hours">-3 hours</div>
<div data-value="-12 hours">-12 hours</div>
<div data-value="-1 day">-1 day</div>
root@9deabcca11c3:/workspace/scubaduck# sed -n '240,280p' scubaduck/static/index
.html
<div data-value="-1 day">-1 day</div>
<div data-value="-3 days">-3 days</div>
<div data-value="-1 week">-1 week</div>
<div data-value="-1 fortnight">-1 fortnight</div>
<div data-value="-30 days">-30 days</div>
<div data-value="-90 days">-90 days</div>
</div>
</div>
</div>
<div class="field">
<label>Order By<span class="help" title="Choose a column to sort resul
ts by.">[?]</span></label>
<select id="order_by"></select>
<button id="order_dir" type="button">ASC \u25B2</button>
</div>
<div class="field">
<label>Limit<span class="help" title="Choose the maximum number of res
ults to show in the chart after any aggregations have been applied. For example,
a limit of 10 will show no more than 10 rows for a table, etc.">[?]</span></lab
el>
<input id="limit" type="number" value="100" />
</div>
<div id="x_axis_field" class="field" style="display:none;">
<label>X-axis</label>
<select id="x_axis"></select>
</div>
<div id="granularity_field" class="field" style="display:none;">
<label>Granularity</label>
<select id="granularity">
<option>Auto</option>
<option>Fine</option>
<option>1 second</option>
<option>5 seconds</option>
<option>10 seconds</option>
<option>30 seconds</option>
<option>1 minute</option>
<option>4 minutes</option>
<option>5 minutes</option>
<option>10 minutes</option>
<option>15 minutes</option>
<option>30 minutes</option>
<option>1 hour</option>
<option>3 hours</option>
<option>6 hours</option>
<option>1 day</option>
I’m checking if the "granularity" select uses the custom dropdown, starting with 'initDropdown' search.
root@9deabcca11c3:/workspace/scubaduck# grep -n "initDropdown" -R .
./scubaduck/static/js/view_settings.js:36:function initDropdown(select) {
./scubaduck/static/js/view_settings.js:353: initDropdown(orderSelect);
./scubaduck/static/js/view_settings.js:354: initDropdown(document.getElementB
yId('aggregate'));
./scubaduck/static/js/view_settings.js:367: initDropdown(tableSel);
./scubaduck/static/js/view_settings.js:534: initDropdown(colSel);
The script uses initDropdown
for multiple selects, though I’ll need to check further around line 350.
sed -n '340,370p' scubaduck/static/js/view_settings.js
root@9deabcca11c3:/workspace/scubaduck# sed -n '340,370p' scubaduck/static/js/vi
ew_settings.js
document.getElementById('columns_none').addEventListener('click', e => {
e.preventDefault();
groupsEl.querySelectorAll('input').forEach(cb => (cb.checked = false));
updateSelectedColumns();
});
updateSelectedColumns();
columnValues.samples = allColumns.slice();
columnValues.table = [];
columnValues.timeseries = [];
groupBy = document.getElementById('group_by').closest('.field');
initChipInput(groupBy, typed =>
allColumns.filter(c => c.toLowerCase().includes(typed.toLowerCase()))
);
initDropdown(orderSelect);
initDropdown(document.getElementById('aggregate'));
});
}
let columnsInitialized = false;
fetch('/api/tables').then(r => r.json()).then(tables => {
const tableSel = document.getElementById('table');
tables.forEach(t => {
const o = document.createElement('option');
o.value = t;
o.textContent = t;
tableSel.appendChild(o);
});
initDropdown(tableSel);
const measure = document.createElement('span');
measure.style.visibility = 'hidden';
measure.style.position = 'absolute';
I'm continuing to scroll to capture more information.
root@9deabcca11c3:/workspace/scubaduck# sed -n '370,410p' scubaduck/static/js/vi
ew_settings.js
measure.style.position = 'absolute';
document.body.appendChild(measure);
let maxWidth = 0;
tables.forEach(t => {
measure.textContent = t;
const w = measure.getBoundingClientRect().width;
if (w > maxWidth) maxWidth = w;
});
measure.remove();
const disp = tableSel.parentElement.querySelector('.dropdown-display');
if (disp) disp.style.minWidth = maxWidth + 30 + 'px';
const table = parseSearch().table || tables[0];
tableSel.value = table;
tableSel.dispatchEvent(new Event('change'));
loadColumns(table).then(() => {
updateDisplayTypeUI();
addFilter();
initFromUrl();
columnsInitialized = true;
});
tableSel.addEventListener('change', () => {
loadColumns(tableSel.value).then(() => {
if (columnsInitialized) {
resetViewSettings();
applyParams({table: tableSel.value});
}
});
});
});
document.querySelectorAll('#tabs .tab').forEach(btn => {
btn.addEventListener('click', () => {
document.querySelectorAll('#tabs .tab').forEach(t => t.classList.remove('act
ive'));
document.querySelectorAll('.tab-content').forEach(c => c.classList.remove('a
ctive'));
btn.classList.add('active');
document.getElementById(btn.dataset.tab).classList.add('active');
});
});
document.querySelectorAll('.rel-btn').forEach(btn => {
btn.addEventListener('click', () => {
I'm continuing to scroll through the content, diving a bit deeper.
root@9deabcca11c3:/workspace/scubaduck# sed -n '410,470p' scubaduck/static/js/vi
ew_settings.js
btn.addEventListener('click', () => {
const dd = document.getElementById(btn.dataset.target);
const show = dd.style.display === 'none' || dd.style.display === '';
document.querySelectorAll('.rel-dropdown').forEach(d => (d.style.display = '
none'));
dd.style.display = show ? 'block' : 'none';
});
});
document.querySelectorAll('.rel-dropdown div').forEach(opt => {
opt.addEventListener('click', () => {
const box = opt.closest('.rel-box');
const input = box.querySelector('input');
input.value = opt.dataset.value || opt.textContent;
opt.parentElement.style.display = 'none';
});
});
document.addEventListener('click', e => {
document.querySelectorAll('.rel-dropdown').forEach(dd => {
if (!dd.parentElement.contains(e.target)) dd.style.display = 'none';
});
});
function updateColumnsTabCount() {
const baseCount = document.querySelectorAll('#column_groups input:checked').le
ngth;
const derivedCount = document.querySelectorAll('#derived_list .derived .d-use:
checked').length;
const btn = document.getElementById('columns_tab');
if (btn) btn.textContent = `Columns (${baseCount + derivedCount})`;
}
function updateSelectedColumns(type = graphTypeSel.value) {
const base = allColumns.filter(name => {
const cb = document.querySelector(`#column_groups input[value="${name}"]`);
if (!cb || !cb.checked) return false;
if (type === 'table' && isStringColumn(name)) return false;
return true;
});
if (type === 'table' || type === 'timeseries') {
selectedColumns = groupBy.chips.slice();
if (document.getElementById('show_hits').checked) selectedColumns.push('Hits
');
const agg = document.getElementById('aggregate').value.toLowerCase();
if (!(type === 'table' && agg === 'count')) {
base.forEach(c => {
if (!selectedColumns.includes(c)) selectedColumns.push(c);
});
derivedColumns.forEach(dc => {
if (dc.include && !selectedColumns.includes(dc.name)) selectedColumns.pu
sh(dc.name);
});
}
} else {
selectedColumns = base.slice();
derivedColumns.forEach(dc => {
if (dc.include) selectedColumns.push(dc.name);
});
}
columnValues[type] = selectedColumns.slice();
const orderCol = document.getElementById('order_by').value;
if (orderCol && !selectedColumns.includes(orderCol)) {
selectedColumns.push(orderCol);
}
updateColumnsTabCount();
}
I'm going further down to see what's at the bottom.
ew_settings.js
}
function isStringColumn(name) {
const t = (columnTypes[name] || '').toUpperCase();
return t.includes('CHAR') || t.includes('STRING') || t.includes('VARCHAR');
}
function isIntegerColumn(name) {
const t = (columnTypes[name] || '').toUpperCase();
return t.includes('INT');
}
function isTimeColumn(name) {
const t = (columnTypes[name] || '').toUpperCase();
if (t.includes('TIMESTAMP') || t.includes('DATE') || t.includes('TIME')) retur
n true;
const sel = document.getElementById('time_column').value;
const xsel = document.getElementById('x_axis').value;
if (name === sel || name === xsel) return true;
return false;
}
function formatNumber(val) {
if (typeof val !== 'number') val = Number(val);
if (Number.isNaN(val)) return '';
if (val === 0) return '0';
const abs = Math.abs(val);
if (abs > 999.999) {
const units = [
{n: 1e12, s: 'T'},
{n: 1e9, s: 'B'},
{n: 1e6, s: 'M'},
{n: 1e3, s: 'K'},
];
for (const u of units) {
if (abs >= u.n) {
return (val / u.n).toFixed(2) + ' ' + u.s;
}
}
}
if (abs < 0.0005) return '0.000';
if (Number.isInteger(val)) return val.toString();
return val.toFixed(3);
}
function addFilter() {
const container = document.createElement('div');
container.className = 'filter';
container.innerHTML = `
<div class="filter-row">
<select class="f-col"></select>
<select class="f-op"></select>
<button type="button" class="remove" onclick="this.closest('.filter').remo
ve()">✖</button>
</div>
<div class="chip-box">
<div class="chip-input">
<input class="f-val" type="text">
<button type="button" class="chip-copy">⎘</button>
</div>
<div class="chip-dropdown"></div>
</div>
`;
const colSel = container.querySelector('.f-col');
colSel.innerHTML = allColumns.map(c => `<option value="${c}">${c}</option>`).j
oin('');
initDropdown(colSel);
function populateOps() {
const opSel = container.querySelector('.f-op');
const col = colSel.value;
const ops = isStringColumn(col)
? [
['=', '='],
['!=', '!='],
['~', 'matches regex'],
['!~', 'not matches regex'],
['contains', 'contains'],
['!contains', 'not contains'],
['empty', 'empty'],
['!empty', 'not empty'],
['LIKE', 'like'],
]
: [
['=', '='],
['!=', '!='],
['<', '<'],
['>', '>'],
];
opSel.innerHTML = ops.map(o => `<option value="${o[0]}">${o[1]}</option>`).j
oin('');
updateInputVis();
}
I'm looking toward the bottom of the file for the section I need.
root@9deabcca11c3:/workspace/scubaduck# sed -n '560,620p' scubaduck/static/js/vi
ew_settings.js
function updateInputVis() {
const op = container.querySelector('.f-op').value;
const box = container.querySelector('.chip-box');
box.style.display = op === 'empty' || op === '!empty' ? 'none' : 'block';
}
colSel.addEventListener('change', populateOps);
container.querySelector('.f-op').addEventListener('change', updateInputVis);
populateOps();
document.getElementById('filter_list').appendChild(container);
initChipInput(container, (typed, el) => {
const colEl = el.querySelector('.f-col select') || el.querySelector('.f-col'
);
if (!colEl) return [];
const col = colEl.value;
if (!isStringColumn(col)) return [];
return fetch(`/api/samples?column=${encodeURIComponent(col)}&q=${encodeURICo
mponent(typed)}`)
.then(r => r.json());
});
}
function nextDerivedName() {
let n = 1;
while (true) {
const name = `derived_${n}`;
if (!derivedColumns.some(d => d.name === name) && !allColumns.includes(name)
) return name;
n++;
}
}
function addDerived(data = {}) {
const container = document.createElement('div');
container.className = 'derived';
container.innerHTML = `
<div class="derived-row">
<select class="d-type">
<option value="aggregated">Aggregated</option>
<option value="string">String</option>
<option value="numeric">Numeric</option>
</select>
<input class="d-name" type="text">
<button type="button" class="remove" onclick="removeDerived(this)">✖</butt
on>
</div>
<label><input type="checkbox" class="d-use" checked> Include in Query</label
>
<textarea class="d-expr" rows="2"></textarea>
`;
document.getElementById('derived_list').appendChild(container);
const obj = {
type: data.type || 'string',
name: data.name || nextDerivedName(),
expr: data.expr || '',
include: data.include !== undefined ? data.include : true,
el: container
};
container.querySelector('.d-type').value = obj.type;
container.querySelector('.d-name').value = obj.name;
container.querySelector('.d-expr').value = obj.expr;
container.querySelector('.d-use').checked = obj.include;
['change','input'].forEach(evt => {
container.addEventListener(evt, refreshDerivedColumns);
});
I’m going to scroll down further to line 620.
root@9deabcca11c3:/workspace/scubaduck# sed -n '620,680p' scubaduck/static/js/vi
ew_settings.js
});
derivedColumns.push(obj);
refreshDerivedColumns();
}
function removeDerived(btn) {
const el = btn.closest('.derived');
const idx = derivedColumns.findIndex(d => d.el === el);
if (idx !== -1) {
derivedColumns.splice(idx, 1);
}
el.remove();
refreshDerivedColumns();
}
function refreshDerivedColumns() {
allColumns.splice(0, allColumns.length, ...baseColumns);
stringColumns.splice(0, stringColumns.length, ...baseStringColumns);
integerColumns.splice(0, integerColumns.length, ...baseIntegerColumns);
timeColumns.splice(0, timeColumns.length, ...baseTimeColumns);
timeColumnOptions.splice(0, timeColumnOptions.length, ...baseTimeColumnOptions
);
derivedColumns.forEach(d => {
d.type = d.el.querySelector('.d-type').value;
d.name = d.el.querySelector('.d-name').value;
d.expr = d.el.querySelector('.d-expr').value;
d.include = d.el.querySelector('.d-use').checked;
allColumns.push(d.name);
columnTypes[d.name] = d.type;
if (d.type === 'string') {
stringColumns.push(d.name);
} else {
integerColumns.push(d.name);
timeColumnOptions.push(d.name);
}
});
updateSelectedColumns();
}
let lastQueryTime = 0;
let queryStart = 0;
function dive(push=true) {
const params = collectParams();
if (push) {
history.pushState(params, '', paramsToSearch(params));
}
const payload = Object.assign({}, params);
const dcMap = {};
(params.derived_columns || []).forEach(d => {
if (d.include) dcMap[d.name] = d.expr;
});
payload.derived_columns = dcMap;
const view = document.getElementById('view');
view.innerHTML = '<p>Loading...</p>';
window.lastResults = undefined;
queryStart = performance.now();
fetch('/api/query', {method:'POST', headers:{'Content-Type':'application/json'
}, body:JSON.stringify(payload)})
.then(async r => {
const data = await r.json();
if (!r.ok) throw data;
return data;
I'll revisit lines near 60 to 78 for context around the open() modification.
root@9deabcca11c3:/workspace/scubaduck# nl -ba scubaduck/static/js/view_settings
.js | sed -n '40,80p'
40 if (disp) {
41 const opt = select.options[select.selectedIndex];
42 disp.textContent = opt ? opt.textContent : '';
43 }
44 return;
45 }
46 select.dataset.dropdownInit = '1';
47
48 const wrapper = document.createElement('div');
49 wrapper.className = 'dropdown';
50 if (select.classList.contains('f-col')) {
51 wrapper.classList.add('f-col');
52 }
53 select.parentNode.insertBefore(wrapper, select);
54 wrapper.appendChild(select);
55 select.style.display = 'none';
56 const disp = document.createElement('div');
57 disp.className = 'dropdown-display';
58 function updateDisplay() {
59 const opt = select.options[select.selectedIndex];
60 disp.textContent = opt ? opt.textContent : '';
61 }
62 updateDisplay();
63 wrapper.appendChild(disp);
64 const menu = document.createElement('div');
65 menu.className = 'dropdown-menu';
66 const search = document.createElement('input');
67 search.placeholder = 'Search';
68 menu.appendChild(search);
69 const list = document.createElement('div');
70 menu.appendChild(list);
71 wrapper.appendChild(menu);
72
73 function close() {
74 menu.style.display = 'none';
75 }
76
77 function open() {
78 renderOptions();
79 menu.style.display = 'block';
80 search.focus();
I'll check lines 80 to 100 to review the content.
root@9deabcca11c3:/workspace/scubaduck# nl -ba scubaduck/static/js/view_settings
.js | sed -n '80,120p'
80 search.focus();
81 }
82
83 disp.addEventListener('click', () => {
84 if (menu.style.display === 'block') {
85 close();
86 } else {
87 open();
88 }
89 });
90
91 document.addEventListener('click', e => {
92 if (!wrapper.contains(e.target)) {
93 close();
94 }
95 });
96
97 function renderOptions() {
98 const q = search.value.toLowerCase();
99 list.innerHTML = '';
100 Array.from(select.options).forEach(o => {
101 if (!o.textContent.toLowerCase().includes(q)) return;
102 const div = document.createElement('div');
103 div.className = 'option';
104 if (q) {
105 const text = o.textContent;
106 const idx = text.toLowerCase().indexOf(q);
107 if (idx !== -1) {
108 div.innerHTML =
109 text.slice(0, idx) +
110 '<u>' +
111 text.slice(idx, idx + q.length) +
112 '</u>' +
113 text.slice(idx + q.length);
114 } else {
115 div.textContent = text;
116 }
117 } else {
118 div.textContent = o.textContent;
119 }
120 if (o.value === select.value) div.classList.add('selected');
root@9deabcca11c3:/workspace/scubaduck# nl -ba scubaduck/static/js/view_settings
.js | sed -n '120,160p'
120 if (o.value === select.value) div.classList.add('selected');
121 div.addEventListener('mousedown', evt => {
122 evt.preventDefault();
123 select.value = o.value;
124 select.dispatchEvent(new Event('change'));
125 updateDisplay();
126 close();
127 });
128 list.appendChild(div);
129 });
130 }
131
132 search.addEventListener('input', renderOptions);
133 select.addEventListener('change', updateDisplay);
134 }
135 let orderDir = 'ASC';
136 const orderDirBtn = document.getElementById('order_dir');
137 const graphTypeSel = document.getElementById('graph_type');
138 function updateOrderDirButton() {
139 orderDirBtn.textContent = orderDir + (orderDir === 'ASC' ? ' \u25B2' :
' \u25BC');
140 }
141
142 function updateDisplayTypeUI() {
143 const prevType = displayType;
144 updateSelectedColumns(prevType);
145 const newType = graphTypeSel.value;
146 const showTable = newType === 'table';
147 const showTS = newType === 'timeseries';
148 document.getElementById('group_by_field').style.display = showTable ||
showTS ? 'flex' : 'none';
149 document.getElementById('aggregate_field').style.display = showTable |
| showTS ? 'flex' : 'none';
150 document.getElementById('show_hits_field').style.display = showTable ?
'flex' : 'none';
151 document.getElementById('x_axis_field').style.display = showTS ? 'flex
' : 'none';
152 document.getElementById('granularity_field').style.display = showTS ?
'flex' : 'none';
153 document.getElementById('fill_field').style.display = showTS ? 'flex'
: 'none';
154 document.querySelectorAll('#column_groups .col-group').forEach(g => {
155 if (g.querySelector('.col-group-header').textContent.startsWith('Str
ings')) {
156 g.style.display = showTable || showTS ? 'none' : '';
157 }
158 });
159 limitValues[prevType] = parseInt(limitInput.value, 10);
160 if (showTS && limitValues.timeseries === undefined) {
I'm patching the open() function between lines 77 and 80, adjusting scroll position correctly.
root@9deabcca11c3:/workspace/scubaduck# cat pyproject.toml
[project]
name = "scubaduck"
version = "0.1.0"
description = "Scuba on DuckDB"
readme = "README.md"
requires-python = ">=3.12"
license = {text = "MIT"}
dependencies = [
"duckdb",
"flask",
"python-dateutil>=2.9.0.post0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.pytest.ini_options]
addopts = "-n auto --tb=native"
[tool.pyright]
# Pyright configuration with strict settings
include = ["scubaduck", "tests"]
exclude = ["**/__pycache__", "dist"]
stubPath = "stubs"
venvPath = "."
venv = ".venv"
reportMissingImports = true
reportMissingTypeStubs = true
pythonVersion = "3.12"
pythonPlatform = "All"
typeCheckingMode = "strict"
reportUnknownMemberType = true
reportUnknownParameterType = true
reportUnknownVariableType = true
reportUnknownArgumentType = true
reportPrivateImportUsage = true
reportUntypedFunctionDecorator = true
reportFunctionMemberAccess = true
reportIncompatibleMethodOverride = true
[tool.ruff]
target-version = "py312"
[tool.uv]
dev-dependencies = [
"pytest>=8.3.5",
"ruff>=0.11.2",
"pytest-xdist>=3.6.1",
"playwright>=1.52.0",
"pyright>=1.1.350",
"pytest-playwright>=0.7.0",
]
The instructions mention ruff check, ruff format, and pyright, but not pytest. I'll follow those.
root@9deabcca11c3:/workspace/scubaduck# git status --short
I'll activate the virtual environment before making any changes.
root@9deabcca11c3:/workspace/scubaduck# source .venv/bin/activate
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# apply_patch<<'PATCH'
> *** Begin Patch
> *** Update File: scubaduck/static/js/view_settings.js
> @@
> function open() {
> renderOptions();
> menu.style.display = 'block';
> + const selected = list.querySelector('.selected');
> + if (selected) {
> + const offset = selected.offsetTop - search.offsetHeight - 4;
> + menu.scrollTop = offset > 0 ? offset : 0;
> + }
> search.focus();
> }
> *** End Patch
> PATCH
Success. Updated the following files:
M scubaduck/static/js/view_settings.js
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll add a test for the dropdown scroll, focusing on the orderSelect
dropdown.
I can't access the network, so I'll test the scroll without verifying table count.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "create table" -R sc
ubaduck | head
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'm checking if the dataset gets inserted through our test file test_server_time.py.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "table" -n tests/tes
t_server.py
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
The dataset might be small; I could modify it or check dropdown options for overflow.
from typing import Any
import pytest
from werkzeug.serving import make_server
from scubaduck.server import create_app
from tests.web_utils import select_value
@pytest.fixture()
def multi_table_server_url() -> Iterator[str]:
app = create_app("TEST")
httpd = make_server("127.0.0.1", 0, app)
port = httpd.server_port
thread = threading.Thread(target=httpd.serve_forever)
thread.start()
try:
yield f"http://127.0.0.1:{port}"
finally:
httpd.shutdown()
thread.join()
def test_table_param_updates_on_dive(page: Any, multi_table_server_url: str) ->
None:
page.goto(multi_table_server_url + "?table=events")
page.wait_for_selector("#table option", state="attached")
select_value(page, "#table", "extra")
page.evaluate("window.lastResults = undefined")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
table_param = page.evaluate(
"new URLSearchParams(window.location.search).get('table')"
)
assert table_param == "extra"
def test_table_dropdown_persists_on_refresh(
page: Any, multi_table_server_url: str
) -> None:
page.goto(multi_table_server_url + "?table=events")
page.wait_for_selector("#table option", state="attached")
select_value(page, "#table", "extra")
page.evaluate("window.lastResults = undefined")
page.click("text=Dive")
page.wait_for_function("window.lastResults !== undefined")
page.reload()
page.wait_for_selector("#table option", state="attached")
assert page.input_value("#table") == "extra"
disp = page.text_content("#table + .dropdown-display")
assert disp is not None and disp.strip() == "extra"
def test_table_switch_resets_view_settings(
page: Any, multi_table_server_url: str
) -> None:
page.goto(multi_table_server_url + "?table=events")
page.wait_for_selector("#table option", state="attached")
page.click("text=Columns")
page.wait_for_selector("#column_groups input", state="attached")
page.uncheck("#column_groups input:first-of-type")
page.click("text=View Settings")
select_value(page, "#graph_type", "table")
page.fill("#limit", "50")
page.evaluate("g => { groupBy.chips = ['name']; groupBy.renderChips(); }")
select_value(page, "#table", "extra")
page.wait_for_function("document.querySelector('#table').value === 'extra'")
assert page.input_value("#graph_type") == "samples"
assert page.input_value("#limit") == "100"
chips = page.evaluate("groupBy.chips.length")
assert chips == 0
page.click("text=Columns")
page.wait_for_selector("#column_groups input", state="attached")
count = page.evaluate(
"document.querySelectorAll('#column_groups input:checked').length"
)
assert count == 3
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'm going to scroll further down to reach the bottom.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# sed -n '120,200p' tests/test
_multi_table_web.py
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
Hmm, I’ll check server.create_app to better understand how it handles the "TEST" environment.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "TEST" -R scubaduck
scubaduck/server.py:406: if isinstance(db_file, str) and db_file.upper() == "
TEST":
server.py
lines.append("GROUP BY " + ", ".join(_quote(c) for c in group_cols))
if order_by:
lines.append(f"ORDER BY {_quote(order_by)} {params.order_dir}")
elif params.graph_type == "timeseries":
lines.append("ORDER BY bucket")
if params.limit is not None:
lines.append(f"LIMIT {params.limit}")
return "\n".join(lines)
def create_app(db_file: str | Path | None = None) -> Flask:
app = Flask(__name__, static_folder="static")
if db_file is None:
env_db = os.environ.get("SCUBADUCK_DB")
if env_db:
db_file = env_db
if isinstance(db_file, str) and db_file.upper() == "TEST":
con = _create_test_database()
else:
db_path = Path(db_file or Path(__file__).with_name("sample.csv")).resolv
e()
con = _load_database(db_path)
tables = [r[0] for r in con.execute("SHOW TABLES").fetchall()]
if not tables:
raise ValueError("No tables found in database")
default_table = tables[0]
columns_cache: Dict[str, Dict[str, str]] = {}
def get_columns(table: str) -> Dict[str, str]:
if table not in columns_cache:
rows = con.execute(f'PRAGMA table_info("{table}")').fetchall()
if not rows:
raise ValueError(f"Unknown table: {table}")
columns_cache[table] = {r[1]: r[2] for r in rows}
return columns_cache[table]
sample_cache: Dict[Tuple[str, str, str], Tuple[List[str], float]] = {}
CACHE_TTL = 60.0
CACHE_LIMIT = 200
@app.route("/")
def index() -> Any: # pyright: ignore[reportUnusedFunction]
assert app.static_folder is not None
return send_from_directory(app.static_folder, "index.html")
@app.route("/js/<path:filename>")
def js(filename: str) -> Any: # pyright: ignore[reportUnusedFunction]
assert app.static_folder is not None
folder = Path(app.static_folder) / "js"
return send_from_directory(folder, filename)
@app.route("/api/tables")
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# sed -n '440,520p' scubaduck/
server.py
@app.route("/api/tables")
def tables_endpoint() -> Any: # pyright: ignore[reportUnusedFunction]
return jsonify(tables)
@app.route("/api/columns")
def columns() -> Any: # pyright: ignore[reportUnusedFunction]
table = request.args.get("table", default_table)
rows = con.execute(f'PRAGMA table_info("{table}")').fetchall()
return jsonify([{"name": r[1], "type": r[2]} for r in rows])
def _cache_get(key: Tuple[str, str, str]) -> List[str] | None:
item = sample_cache.get(key)
if item is None:
return None
vals, ts = item
if time.time() - ts > CACHE_TTL:
del sample_cache[key]
return None
sample_cache[key] = (vals, time.time())
return vals
def _cache_set(key: Tuple[str, str, str], vals: List[str]) -> None:
sample_cache[key] = (vals, time.time())
if len(sample_cache) > CACHE_LIMIT:
oldest = min(sample_cache.items(), key=lambda kv: kv[1][1])[0]
del sample_cache[oldest]
@app.route("/api/samples")
def sample_values() -> Any: # pyright: ignore[reportUnusedFunction]
table = request.args.get("table", default_table)
column = request.args.get("column")
substr = request.args.get("q", "")
column_types = get_columns(table)
if not column or column not in column_types:
return jsonify([])
ctype = column_types[column].upper()
if "CHAR" not in ctype and "STRING" not in ctype and "VARCHAR" not in ct
ype:
return jsonify([])
key = (table, column, substr)
cached = _cache_get(key)
if cached is not None:
return jsonify(cached)
qcol = _quote(column)
rows = con.execute(
f"SELECT DISTINCT {qcol} FROM \"{table}\" WHERE CAST({qcol} AS VARCH
AR) ILIKE '%' || ? || '%' LIMIT 20",
[substr],
).fetchall()
values = [r[0] for r in rows]
_cache_set(key, values)
return jsonify(values)
@app.route("/api/query", methods=["POST"])
def query() -> Any: # pyright: ignore[reportUnusedFunction]
payload = request.get_json(force=True)
try:
start = parse_time(payload.get("start"))
end = parse_time(payload.get("end"))
except Exception as exc:
return jsonify({"error": str(exc)}), 400
params = QueryParams(
start=start,
end=end,
order_by=payload.get("order_by"),
order_dir=payload.get("order_dir", "ASC"),
limit=payload.get("limit"),
columns=payload.get("columns", []),
derived_columns=payload.get("derived_columns", {}),
graph_type=payload.get("graph_type", "samples"),
group_by=payload.get("group_by", []),
aggregate=payload.get("aggregate"),
show_hits=payload.get("show_hits", False),
x_axis=payload.get("x_axis"),
granularity=payload.get("granularity", "Auto"),
fill=payload.get("fill", "0"),
table=payload.get("table", default_table),
time_column=payload.get("time_column", "timestamp"),
time_unit=payload.get("time_unit", "s"),
)
if params.order_by == "Samples":
params.order_by = "Hits"
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I’ll dig into the _create_test_database function now. Let's see what it does.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "_create_test_databa
se" -n scubaduck/server.py
82:def _create_test_database() -> duckdb.DuckDBPyConnection:
407: con = _create_test_database()
erver.py
r[0]
for r in con.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()
]
for t in tables:
con.execute(f'CREATE VIEW "{t}" AS SELECT * FROM db."{t}"')
else:
con = duckdb.connect(path)
return con
def _create_test_database() -> duckdb.DuckDBPyConnection:
"""Return a DuckDB connection with a small multi-table dataset."""
con = duckdb.connect()
con.execute(
"CREATE TABLE events (id INTEGER PRIMARY KEY, ts INTEGER, val REAL, name
TEXT, flag BOOLEAN)"
)
con.execute("INSERT INTO events VALUES (1, 1704067200, 1.5, 'alice', 1)")
con.execute("INSERT INTO events VALUES (2, 1704070800, 2.0, 'bob', 0)")
con.execute('CREATE TABLE extra (ts INTEGER, "desc" TEXT, num INTEGER)')
con.execute("INSERT INTO extra VALUES (1704067200, 'x', 1)")
con.execute("INSERT INTO extra VALUES (1704070800, 'y', 2)")
return con
_REL_RE = re.compile(
r"([+-]?\d+(?:\.\d*)?)\s*(hour|hours|day|days|week|weeks|fortnight|fortnight
s|month|months|year|years)",
re.IGNORECASE,
)
def parse_time(val: str | None) -> str | None:
"""Parse an absolute or relative time string into ``YYYY-MM-DD HH:MM:SS``.""
"
if val is None or val == "":
return None
s = val.strip()
if s.lower() == "now":
dt = datetime.now(timezone.utc)
return dt.replace(microsecond=0).strftime("%Y-%m-%d %H:%M:%S")
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
The dataset only has two tables. I'll verify the "granularity" field in JS next.
157: document.getElementById('granularity_field').style.display = showTS ? 'fle
x' : 'none';
739: payload.granularity = document.getElementById('granularity').value;
768: if (params.granularity) sp.set('granularity', params.granularity);
800: if (params.granularity) document.getElementById('granularity').value = par
ams.granularity;
851: document.getElementById('granularity').value = 'Auto';
899: if (sp.has('granularity')) params.granularity = sp.get('granularity');
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "granularity" scubad
uck/static/js/view_settings.js
157: document.getElementById('granularity_field').style.display = showTS ? 'fle
x' : 'none';
739: payload.granularity = document.getElementById('granularity').value;
768: if (params.granularity) sp.set('granularity', params.granularity);
800: if (params.granularity) document.getElementById('granularity').value = par
ams.granularity;
851: document.getElementById('granularity').value = 'Auto';
899: if (sp.has('granularity')) params.granularity = sp.get('granularity');
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll modify the test to add options and check that the selected item is near the top.
I'll add the test after "test_table_selector_dropdown," ensuring the patch is effective.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "test_table_selector
_dropdown" -n tests/test_web_basics.py
92:def test_table_selector_dropdown(page: Any, server_url: str) -> None:
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll begin by checking around here.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# sed -n '92,112p' tests/test_
web_basics.py
def test_table_selector_dropdown(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#table option", state="attached")
disp = page.query_selector("#table + .dropdown-display")
assert disp
assert (
page.evaluate("getComputedStyle(document.querySelector('#table')).displa
y")
== "none"
)
assert page.query_selector("#table + .dropdown-display + .dropdown-menu inpu
t")
def test_x_axis_default_entry(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#graph_type", state="attached")
select_value(page, "#graph_type", "timeseries")
page.wait_for_selector("#x_axis option", state="attached")
options = page.locator("#x_axis option").all_inner_texts()
assert "(default)" in options
assert page.input_value("#x_axis") == ""
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I’ll scroll down to check on test_end_dropdown_now and related items.
text_align = page.evaluate(
"getComputedStyle(document.querySelector('#settings label')).textAlign"
)
assert text_align == "right"
def test_relative_dropdown(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
btn = page.query_selector('[data-target="start-select"]')
assert btn
btn.click()
page.click("#start-select div:text('-3 hours')")
assert page.input_value("#start") == "-3 hours"
def test_end_dropdown_now(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.click('[data-target="end-select"]')
page.click("#end-select div:text('now')")
assert page.input_value("#end") == "now"
def test_invalid_time_error_shown(page: Any, server_url: str) -> None:
data = run_query(
page,
server_url,
start="nonsense",
end="now",
order_by="user",
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll add the new test after existing ones. Need to check for setSelectValue
function first.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# grep -n "setSelectValue" -R
scubaduck/static
scubaduck/static/js/view_settings.js:920:function setSelectValue(selector, value
) {
if (sp.has('fill')) params.fill = sp.get('fill');
if (sp.has('derived_columns')) {
try { params.derived_columns = JSON.parse(sp.get('derived_columns')); } catc
h(e) { params.derived_columns = []; }
}
return params;
}
function initFromUrl() {
const params = parseSearch();
history.replaceState(params, '', paramsToSearch(params));
applyParams(params);
dive(false);
}
window.addEventListener('popstate', e => {
const params = e.state || parseSearch();
applyParams(params);
dive(false);
});
function setSelectValue(selector, value) {
const el = typeof selector === 'string' ? document.querySelector(selector) : s
elector;
if (el) {
const select = el.tagName === 'SELECT' ? el : el.querySelector('select');
if (select) {
select.value = value;
select.dispatchEvent(new Event('change'));
}
}
}
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I’ll use setSelectValue in page.evaluate, but avoid replicating ephemeral behavior. Time to patch.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# apply_patch<<'PATCH'
> *** Begin Patch
> *** Update File: tests/test_web_basics.py
> @@
> def test_table_selector_dropdown(page: Any, server_url: str) -> None:
> page.goto(server_url)
> page.wait_for_selector("#table option", state="attached")
> disp = page.query_selector("#table + .dropdown-display")
> @@
> assert page.query_selector("#table + .dropdown-display + .dropdown-menu i
nput")
> +
> +
> +def test_dropdown_scroll_to_selected(page: Any, server_url: str) -> None:
> + page.goto(server_url)
> + page.wait_for_selector("#table option", state="attached")
> + page.evaluate(
> + "() => { const sel=document.getElementById('table'); for(let i=0;i<30
;i++){const o=document.createElement('option'); o.value='t'+i; o.textContent='Ta
ble '+i; sel.appendChild(o);} setSelectValue(sel,'t25'); }"
> + )
> + page.click("#table + .dropdown-display")
> + page.wait_for_selector("#table + .dropdown-display + .dropdown-menu div.s
elected")
> + scroll_top = page.evaluate(
> + "document.querySelector('#table + .dropdown-display + .dropdown-menu'
).scrollTop"
> + )
> + assert scroll_top > 0
> *** End Patch
> PATCH
Success. Updated the following files:
M tests/test_web_basics.py
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll inspect the patch to make sure it's applied properly and avoid any formatting issues.
assert page.query_selector("#table + .dropdown-display + .dropdown-menu inpu
t")
def test_dropdown_scroll_to_selected(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#table option", state="attached")
page.evaluate(
"() => { const sel=document.getElementById('table'); for(let i=0;i<30;i+
+){const o=document.createElement('option'); o.value='t'+i; o.textContent='Table
'+i; sel.appendChild(o);} setSelectValue(sel,'t25'); }"
)
page.click("#table + .dropdown-display")
page.wait_for_selector("#table + .dropdown-display + .dropdown-menu div.sele
cted")
scroll_top = page.evaluate(
"document.querySelector('#table + .dropdown-display + .dropdown-menu').s
crollTop"
)
assert scroll_top > 0
def test_x_axis_default_entry(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#graph_type", state="attached")
select_value(page, "#graph_type", "timeseries")
page.wait_for_selector("#x_axis option", state="attached")
options = page.locator("#x_axis option").all_inner_texts()
assert "(default)" in options
assert page.input_value("#x_axis") == ""
def test_simple_filter(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#order_by option", state="attached")
page.click("text=Add Filter")
filter_el = page.query_selector("#filters .filter:last-child")
assert filter_el
page.evaluate(
"arg => setSelectValue(arg.el.querySelector('.f-col'), arg.val)",
{"el": filter_el, "val": "user"},
)
val_input = filter_el.query_selector(".f-val")
val_input.click()
page.keyboard.type("alice")
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I’ll check the whole patch with ruff format, ruff check, and pyright now.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# ruff format scubaduck/static
/js/view_settings.js tests/test_web_basics.py >/tmp/ruff_format.log && tail -n 2
0 /tmp/ruff_format.log
error: Failed to parse scubaduck/static/js/view_settings.js:1:1: Expected a stat
ement
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
Ruff can't format JS, so I'll only apply it to the Python test file.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# ruff format tests/test_web_b
asics.py >/tmp/ruff_format.log && tail -n 20 /tmp/ruff_format.log
1 file left unchanged
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll go ahead and run the ruff check across the entire repository.
k.log && tail -n 20 /tmp/ruff_check.log
All checks passed!
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# pyright >/tmp/pyright.log &&
tail -n 20 /tmp/pyright.log
0 errors, 0 warnings, 0 informations
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll check the status of the repository now.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git status --short
M scubaduck/static/js/view_settings.js
M tests/test_web_basics.py
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I’ll check the patch for proper formatting, like ensuring there are newlines at the end.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git diff --color -- scubaduc
k/static/js/view_settings.js | sed -n '70,90p'
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
Seems like something got truncated. I'll try showing the diff now.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git diff --color scubaduck/s
tatic/js/view_settings.js | head -n 20
diff --git a/scubaduck/static/js/view_settings.js b/scubaduck/static/js/view_set
tings.js
index 2d26006..c997831 100644
--- a/scubaduck/static/js/view_settings.js
+++ b/scubaduck/static/js/view_settings.js
@@ -77,6 +77,11 @@ function initDropdown(select) {
function open() {
renderOptions();
menu.style.display = 'block';
+ const selected = list.querySelector('.selected');
+ if (selected) {
+ const offset = selected.offsetTop - search.offsetHeight - 4;
+ menu.scrollTop = offset > 0 ? offset : 0;
+ }
search.focus();
}
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll check the diff now to ensure test_web_basics.py is accurate.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git diff --color tests/test_
web_basics.py | head -n 40
diff --git a/tests/test_web_basics.py b/tests/test_web_basics.py
index 56826b7..fa243de 100644
--- a/tests/test_web_basics.py
+++ b/tests/test_web_basics.py
@@ -101,6 +101,20 @@ def test_table_selector_dropdown(page: Any, server_url: str
) -> None:
assert page.query_selector("#table + .dropdown-display + .dropdown-menu inp
ut")
+def test_dropdown_scroll_to_selected(page: Any, server_url: str) -> None:
+ page.goto(server_url)
+ page.wait_for_selector("#table option", state="attached")
+ page.evaluate(
+ "() => { const sel=document.getElementById('table'); for(let i=0;i<30;i
++){const o=document.createElement('option'); o.value='t'+i; o.textContent='Tabl
e '+i; sel.appendChild(o);} setSelectValue(sel,'t25'); }"
+ )
+ page.click("#table + .dropdown-display")
+ page.wait_for_selector("#table + .dropdown-display + .dropdown-menu div.sel
ected")
+ scroll_top = page.evaluate(
+ "document.querySelector('#table + .dropdown-display + .dropdown-menu').
scrollTop"
+ )
+ assert scroll_top > 0
+
+
def test_x_axis_default_entry(page: Any, server_url: str) -> None:
page.goto(server_url)
page.wait_for_selector("#graph_type", state="attached")
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
Time to commit the changes!
view_settings.js tests/test_web_basics.py
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git commit -m "feat(ui): scr
oll dropdown to selected option" && git status --short
[work b1f03fb] feat(ui): scroll dropdown to selected option
2 files changed, 19 insertions(+)
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
It looks like the commit output got cut off. I'll display the full details now.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git status --short
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll verify the commit log to confirm the changes.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git log -1 --stat
commit b1f03fb578e35b7721f5e1e209f85302eae1cb99 (HEAD -> work)
Author: Codex <codex@openai.com>
Date: Fri May 23 02:42:18 2025 +0000
feat(ui): scroll dropdown to selected option
scubaduck/static/js/view_settings.js | 5 +++++
tests/test_web_basics.py | 14 ++++++++++++++
2 files changed, 19 insertions(+)
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
Time to put together the PR message and gather code changes for the JS patch.
iew_settings.js | sed -n '72,85p'
72
73 function close() {
74 menu.style.display = 'none';
75 }
76
77 function open() {
78 renderOptions();
79 menu.style.display = 'block';
80 const selected = list.querySelector('.selected');
81 if (selected) {
82 const offset = selected.offsetTop - search.offsetHeight - 4;
83 menu.scrollTop = offset > 0 ? offset : 0;
84 }
85 search.focus();
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll grab the lines from tests/test_web_basics.py now.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# nl -ba tests/test_web_basics
.py | sed -n '100,128p'
100 )
101 assert page.query_selector("#table + .dropdown-display + .dropdown-m
enu input")
102
103
104 def test_dropdown_scroll_to_selected(page: Any, server_url: str) -> None
:
105 page.goto(server_url)
106 page.wait_for_selector("#table option", state="attached")
107 page.evaluate(
108 "() => { const sel=document.getElementById('table'); for(let i=0
;i<30;i++){const o=document.createElement('option'); o.value='t'+i; o.textConten
t='Table '+i; sel.appendChild(o);} setSelectValue(sel,'t25'); }"
109 )
110 page.click("#table + .dropdown-display")
111 page.wait_for_selector("#table + .dropdown-display + .dropdown-menu
div.selected")
112 scroll_top = page.evaluate(
113 "document.querySelector('#table + .dropdown-display + .dropdown-
menu').scrollTop"
114 )
115 assert scroll_top > 0
116
117
118 def test_x_axis_default_entry(page: Any, server_url: str) -> None:
119 page.goto(server_url)
120 page.wait_for_selector("#graph_type", state="attached")
121 select_value(page, "#graph_type", "timeseries")
122 page.wait_for_selector("#x_axis option", state="attached")
123 options = page.locator("#x_axis option").all_inner_texts()
124 assert "(default)" in options
125 assert page.input_value("#x_axis") == ""
126
127
128 def test_simple_filter(page: Any, server_url: str) -> None:
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll gather the terminal outputs for ruff check and pyright now.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# cat /tmp/ruff_check.log
All checks passed!
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# cat /tmp/pyright.log
0 errors, 0 warnings, 0 informations
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll summarize the changes, noting the environment's limitations and confirming the commit diff.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# git show --stat b1f03fb
commit b1f03fb578e35b7721f5e1e209f85302eae1cb99 (HEAD -> work)
Author: Codex <codex@openai.com>
Date: Fri May 23 02:42:18 2025 +0000
feat(ui): scroll dropdown to selected option
scubaduck/static/js/view_settings.js | 5 +++++
tests/test_web_basics.py | 14 ++++++++++++++
2 files changed, 19 insertions(+)
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll reference the lines in the PR body and mention the new scroll test.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# nl -ba scubaduck/static/js/v
iew_settings.js | sed -n '76,85p'
76
77 function open() {
78 renderOptions();
79 menu.style.display = 'block';
80 const selected = list.querySelector('.selected');
81 if (selected) {
82 const offset = selected.offsetTop - search.offsetHeight - 4;
83 menu.scrollTop = offset > 0 ? offset : 0;
84 }
85 search.focus();
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
I'll grab the specific lines from the tests that are relevant here.
(scubaduck) root@9deabcca11c3:/workspace/scubaduck# nl -ba tests/test_web_basics
.py | sed -n '100,116p'
100 )
101 assert page.query_selector("#table + .dropdown-display + .dropdown-m
enu input")
102
103
104 def test_dropdown_scroll_to_selected(page: Any, server_url: str) -> None
:
105 page.goto(server_url)
106 page.wait_for_selector("#table option", state="attached")
107 page.evaluate(
108 "() => { const sel=document.getElementById('table'); for(let i=0
;i<30;i++){const o=document.createElement('option'); o.value='t'+i; o.textConten
t='Table '+i; sel.appendChild(o);} setSelectValue(sel,'t25'); }"
109 )
110 page.click("#table + .dropdown-display")
111 page.wait_for_selector("#table + .dropdown-display + .dropdown-menu
div.selected")
112 scroll_top = page.evaluate(
113 "document.querySelector('#table + .dropdown-display + .dropdown-
menu').scrollTop"
114 )
115 assert scroll_top > 0
116
(scubaduck) root@9deabcca11c3:/workspace/scubaduck#
Time to gather everything needed and write the PR!