process_tables_expire() computes the next expiration date of a table's
bucket under the bucket's lock, but releases the lock (and frees the
expired entries) before replacing the bucket's next_exp with this date. In
this window, another thread may insert a new entry (under the bucket's
lock) and call stktable_requeue_exp() for it. If next_exp still holds the
old (past) date, stktable_requeue_exp() considers the table as already
queued early enough and does nothing. Then the expiration task replaces
next_exp with the later date it computed, possibly TICK_ETERNITY, and
repositions or removes the table from its tree accordingly. The new entry
then doesn't expire until some other activity on this table requeues it.
The window was widened by commit
3e25104a9 ("MEDIUM: stick-tables: Avoid
freeing elements while holding a lock") which moved the freeing of expired
entries there.
Let's set next_exp before releasing the bucket's lock, so that any
insertion performed after it sees the new value. Since entries are
inserted under this lock, the computed date covers every entry present
in the bucket, so a concurrent stktable_requeue_exp() may safely be
overwritten and a plain store is enough, the CAS is not needed anymore.
This was introduced in 3.3 by commit
8bc8a21b2 ("MEDIUM: stick-tables: Use
a per-shard expiration task") and must be backported to 3.3.
if (!tick_isset(task_exp) || (tick_isset(next_exp_table) && tick_is_lt(next_exp_table, task_exp)))
task_exp = next_exp_table;
+
+ /* under the lock so that later insertions see it */
+ HA_ATOMIC_STORE(&t->buckets[bucket].next_exp, next_exp_table);
HA_RWLOCK_WRUNLOCK(STK_TABLE_LOCK, &t->buckets[bucket].sh_lock);
while (!LIST_ISEMPTY(&tofree_list)) {
tmpnode = eb32_next(table_eb);
if (table_eb->key != next_exp_table) {
- int old_exp;
- /*
- * We have to move the entry in the tree
- */
- old_exp = HA_ATOMIC_LOAD(&t->buckets[bucket].next_exp);
- if (old_exp >= table_eb->key) {
- HA_ATOMIC_CAS(&t->buckets[bucket].next_exp, &old_exp, next_exp_table);
- }
-
/*
* Move the table to its next expiration date, if any.
*/