• docs/v322_new.md webv4/li

    From Rob Swindell (on Windows@VERT to Git commit to main/sbbs/m on Wed Nov 11 13:55:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/c5f0ca87d9b659dae178b1ad
    Modified Files:
    docs/v322_new.md webv4/lib/forum.js
    Log Message:
    webv4: list a user's mail from one index read, not a lookup per number

    get_mail_headers() and getMailHeaders() called MsgBase.get_msg_header()
    for every message number from first_msg to last_msg. On a mail base
    whose numbers are sparse that is far more lookups than messages, and
    each one is a locked index search plus a header read.

    Symptom: the mail page (000-mail.xjs) on vert.synchro.net took 5 min
    51 s to load for a logged-in user. The mail base held 9,262 messages
    numbered 727309-752178, so 24,870 lookups at about 8.6 ms each over
    an SMB share, 15,608 of them for numbers that no longer exist.

    Read the index once with MsgBase.get_index(), keep the records to or
    from the user, and read only those headers (926 here), by index
    offset. The same call now takes 2.3 s. The header's to_ext/from_ext
    are still what decide the listing, so the results are unchanged.

    The per-number loops have been there since webv4 was imported into
    this repository in 3f16cac564 (2020-09-07).

    Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
  • From Rob Swindell (on Windows@VERT to Git commit to main/sbbs/m on Wed Nov 11 22:22:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/dbc2439f2dc687e41c8eaf66
    Modified Files:
    docs/v322_new.md webv4/lib/events/forum.js webv4/lib/forum.js
    Log Message:
    webv4: count unread forum messages from the index, not a lookup per number

    getSubUnreadCount() called MsgBase.get_msg_header() for every message
    number from the user's scan pointer to last_msg. Sparse numbering makes
    that far more lookups than messages, and lib/events/forum.js re-ran it
    for every sub, every refresh_interval (60 s), for each logged-in viewer
    of the forum's group or sub list.

    Symptom: on vert.synchro.net, with the message bases on an SMB share, get-group-unread-counts for a user with 28,900 unread messages took
    355 s, so the 60 s scan ran back to back for as long as the page was
    open. It now takes 9 s; get-sub-unread-counts for one 20-sub group
    went from 69 s to 3 s, with identical output.

    Read the index once with MsgBase.get_index() and walk back from its end
    to the scan pointer. A header is read only when counting messages to
    the user and the index's name CRC matches. A sub whose last_msg,
    total_msgs, scan pointer and scan config are unchanged since it was
    last counted by the same script returns the previous count, so the
    event stream stops recounting idle subs. A message deleted without any
    other change to its sub is not noticed until then.

    In lib/events/forum.js, set last_run after the scan rather than before (fac16e6927, 2019-07-19), so a scan slower than refresh_interval is not followed immediately by another, and don't rebuild a sub's threads when
    its message base and the scan pointer are unchanged.

    Two fixes in getMessageThreads():

    - With forum_no_spam set and max_messages=0 it threw a ReferenceError:
    is_spam() was passed an undefined variable (08a2638b8d, 2020-01-08).
    - With max_messages > 0 it read the numbers last_msg - max through
    last_msg, which hold fewer than max messages when numbering is
    sparse, so threads were dropped (61cb4097fb, 2016-12-25). Take the
    newest max messages from the index.

    The per-number unread loop has been there since webv4 was imported
    into this repository in 3f16cac564 (2020-09-07).

    Part of #1339. Thread views still read every header in the sub.

    Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
  • From Rob Swindell (on Debian L@VERT to Git commit to main/sbbs/m on Fri Nov 13 20:36:00 2026
    https://gitlab.synchro.net/main/sbbs/-/commit/d7197cf6561161858894371f
    Modified Files:
    docs/v322_new.md webv4/lib/forum.js
    Log Message:
    webv4: cache a sub's forum threads instead of rebuilding them per request

    Every thread list, thread view, "load more" and vote refresh rebuilt the
    sub's threads from get_all_msg_headers(), reading every header in the
    sub. Reading the headers is only about a third of that cost: each field
    read from a header object costs about 1 us, so copying 13 fields from
    every message took as long again.

    getMessageThreads() now keeps the result in
    data/webv4/threads/<code>.json. The first line is a key made from the
    .shd and .sid file dates and sizes, max_messages, forum_no_spam and a
    format version, so any post, vote, deletion or header update rebuilds
    it. Nothing per-user is cached; unread counts are computed from the
    user's scan pointer after loading. The file is written a thread at a
    time (one JSON.stringify() of a large sub can exhaust the JS heap) to a temporary file that is then renamed into place, and not at all when the
    base changed within the last 5 seconds, since file dates have
    one-second resolution and hosts sharing the base may disagree on the
    time.

    The cached result is byte-identical to a rebuild. Measured on Linux:

    9,103 messages: 392 ms -> 181 ms (rebuild and write: 630 ms)
    23,248 messages: 925 ms -> 433 ms (1.35 s)
    159,560 messages: 7.1 s -> 2.8 s (10.1 s)

    Part of #1339.

    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>