Please wait...
0 files
Hello,
I installed the following on my Fedora 43 server.
Running: Citadel Server 17846585 with WebCit 17846585
Server Build: 17846585
The DKIM addition worked great. Thanks!
I did however find an issue that I'm hoping can be corrected. Here is what is happening:
I like to separate my spam from my good email so my phone doesn't get overloaded. In the old version I had a rule that looked for the Spam-Status flag and if it was 'Yes' it would forward the email to a spam account and then delete the message. This way the email went to the spam account and cleared up the main account. It also allowed me to go back to that spam account in case email went there by accident.
Here is the problem. When the process is forwarding the message it is running into a lock issue and can't continue. It leaves the message hanging and it never times out. Eventually the ports fill up (even with the threads set to 1024) and the server eventually stops taking calls.
Here is part of a conversation with Gemini that was achieved by running a debugger and tracing the futex to the memory location.
Root Cause Identified: Self-Deadlock in Thread 186
The backtrace identifies Thread 186 (LWP 10186) as the root cause of the entire thread cascade freeze.
The process is suffering from a self-deadlock caused by non-recursive mutex re-entrancy in Citadel's inboxrules module.
The Execution Chain (Thread 186)
-
Housekeeping runs: Thread 186 initiated periodic background processing (
do_housekeeping->perform_inbox_processing). -
Lock acquired:
perform_inbox_processing()locked Critical Section22(S_INBOXRULES). -
Redirect triggered: While iterating through user messages (
for_each_message->inbox_do_msg), it hit an inbox rule configured to redirect an email (inbox_do_redirect). -
Message re-submission:
inbox_do_redirect()calledCtdlSubmitMsg()to resend/route the message into the destination room. -
Room hook fired:
CtdlSubmitMsg()calledPerformRoomHooks()to notify modules that a new message was saved. -
Re-entrant lock attempt:
PerformRoomHooks()invokedserv_inboxrules_roomhook(), which unconditionally attempted to acquirebegin_critical_section(22)again. -
Self-Deadlock: Because Citadel’s
begin_critical_section()mutexes are standard non-recursivepthread_mutex_tlocks, Thread 186 blocked waiting for itself to release critical section 22.
Why All SMTP Worker Threads Stalled
Because Thread 186 froze while holding Critical Section 22:
-
Every incoming SMTP connection (Threads 189, 188, 187, 185, 184, 183, 182, 181, 180, etc.) tried to deliver inbound mail via
CtdlSubmitMsg(). -
Delivery triggers room hooks (
PerformRoomHooks()), which callsserv_inboxrules_roomhook(). -
serv_inboxrules_roomhook()requestedbegin_critical_section(22)and immediately blocked waiting on Thread 186. -
As remote mailers kept attempting delivery, new worker threads spawned, hit the exact same lock, and stacked up until thread limits and socket file descriptors in
CLOSE_WAITwere exhausted.
Remediation Options
1. Immediate Workaround (No Recompile)
If an active account has an Inbox Rule configured to Redirect or Forward messages, removing or disabling that specific rule will prevent inbox_do_redirect() from firing during background housekeeping.
2. Code Fixes (C Source Level)
-
Option A (Unlock before submission): In
server/modules/inboxrules/serv_inboxrules.c, release critical section 22 prior to callinginbox_do_redirect()/CtdlSubmitMsg(), then re-acquire it afterward if still iterating. -
Option B (Re-entrancy Guard): Check if the current thread already holds
S_INBOXRULES(or pass a flag/session context) insideserv_inboxrules_roomhook()before attemptingbegin_critical_section(22). -
Option C (Recursive Mutex): Initialize the mutex for Critical Section 22 with
PTHREAD_MUTEX_RECURSIVEinserver/threads.cso the same thread can safely re-acquire it without deadlocking.
I have removed the server side rules and the email server is working properly. If you would like to duplicate the issue here is what the rules look like:
Thanks in advance for any help on this.