Skip to content

[2.1] Handle failed prepare statements and oversized data in error logs - #9872

Open
jdarwood007 wants to merge 1 commit into
SimpleMachines:release-2.1from
jdarwood007:2.1/errorLogFixes
Open

jdarwood007 wants to merge 1 commit into
SimpleMachines:release-2.1from
jdarwood007:2.1/errorLogFixes

Conversation

@jdarwood007

Copy link
Copy Markdown
Member

Fixes #7831

@albertlast

Copy link
Copy Markdown
Collaborator

I tested this against an installed 2.1 forum on both engines, by calling $smcFunc['db_error_insert'] with 16 kinds of message under three IPs, both normally and with the prepare forced to fail. I have some concerns:

1. The fallback to db_insert. When the statement cannot be prepared, the forum is already in a bad state, and the regular query path may be the very thing that failed. Its own errors are logged by calling smf_db_error_insert() again. #7831 discussed this, and keeping error logging off the normal insert path was the reason for the prepared statement in the first place. release-2.1 already returns when the prepare fails, and this PR removes that guard.

As written, the fallback also fails on both engines:

  • Subs-Db-mysql.php calls $smcFunc['db_insert'] without declaring $smcFunc global, which is a fatal "Value of type null is not callable".
  • Both engines pass a null IP, which {inet:...} rejects ("The database value you're trying to insert does not exist: ip"). A null IP is what every error from cron, the CLI or a background task has. On PostgreSQL that failure is logged through the same path and ends in "Error loop."

2. substr() cuts bytes, not characters. A long message cut inside a multibyte character leaves invalid UTF-8, which is then rejected and the error lost. A 70 KB message of 3-byte characters was not stored on either engine.

3. Pooling on PostgreSQL. The unnamed statement used with $db_persist is now prepared once and cached in $pg_error_data_prep. Any other query may replace the unnamed statement in the meantime, so it needs preparing every time, as release-2.1 does.

normal prepare fails
release-2.1, MySQL 30 / 48 stored 0 / 48 stored
release-2.1, PostgreSQL 45 / 48 stored 0 / 48 stored
this PR, MySQL 36 / 48 stored fatal: $smcFunc is null
this PR, PostgreSQL 42 / 48 stored "Error loop."

I've opened #9883 as an alternative. It keeps to the prepared statement only and keeps the bail-out when the statement cannot be prepared. Before binding, it cleans the values. On a UTF-8 forum, strings are cut with mb_strcut() and invalid UTF-8 is replaced. On MySQL, 4-byte characters also go through fix_utf8mb4, since tables that are not utf8mb4 reject them otherwise. NUL bytes are removed. It stores 48/48 on both engines, and the failing-prepare case finishes without an error or a loop.

@jdarwood007

Copy link
Copy Markdown
Member Author

$smcFunc['db_insert'] tells the error logging to skip when we are inserting into some tables, one of those being the error log. It should not log. However, I added a untested loop detection.

@live627

live627 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

@jdarwood007 jdarwood007 modified the milestones: 2.1.9, 2.1.10 Nov 13, 2026
@albertlast

Copy link
Copy Markdown
Collaborator

I re-ran the same tests against the current head (57654a6), on MySQL and PostgreSQL. Each message ran in a PHP process of its own, so that the new counter doesn't hide anything.

1. Only the first two errors of a request are logged. static $loop goes up on every call and is never reset, so from the third call on, smf_db_error_insert() returns without logging. Logging five ordinary errors in one process:

MySQL PostgreSQL
release-2.1 5 stored 5 stored
this PR 2 stored 2 stored

2. PostgreSQL never uses the prepared statement. Subs-Db-postgresql.php tests $mysql_error_data_prep, which does not exist in that function, so every error goes through db_insert. The normal run and the run with the prepare made to fail gave identical results. Through db_insert:

  • an error without an IP, as every error from cron, the CLI or a background task has, is a fatal "The database value you're trying to insert does not exist: ip";
  • invalid UTF-8 throws ValueError: pg_escape_string(): Argument #2 ($string) Escaping string failed.

3. The MySQL fallback still fails. With the prepare forced to fail (verified before each run), all 48 messages ended in the fatal "The database value you're trying to insert does not exist: ip", for every IP, including a valid IPv4. The IP is converted to hex for the prepared statement before the branch, so db_insert never gets the address as given.

4. $smcFunc['substr'] counts characters, and the columns are limited in bytes. It cuts to 65534 characters, which for 2-, 3- or 4-byte characters is far more than the 65,535 bytes a MySQL TEXT column holds. On invalid UTF-8 its preg_split(…/u) returns false:

MySQL, normal mode, 16 messages × 3 IPs
stored 33 / 48
lost long 2-byte, 3-byte and emoji messages, and short emoji (non-utf8mb4 tables)
invalid UTF-8 TypeError: array_slice(): Argument #1 ($array) must be of type array, bool given, inside the error logger

#9883 avoids all of these, storing 48/48 on both engines. There is no fallback and no counter. Strings are cut with mb_strcut() to the column's size in bytes, invalid bytes are replaced, and fix_utf8mb4 handles emoji for tables that are not utf8mb4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mysqli_prepare can return false

3 participants