Please wait...
0 files
The joy is premature. On the test server of x86 architecture the problem was solved. But on the main working server with aarm64 (Orange Pi 5) on Debian 11 it was only possible to update all components to version 1009, but also all changes lead to a fail and arclog files start multiplying and citadel no longer starts.
Is it possible to create mail filter rules for rooms ?
Greetings
Goose
Armbian is more up to date.
The joy is premature. On the test server of x86 architecture the problem was solved. But on the main working server with aarm64 (Orange Pi 5) on Debian 11 it was only possible to update all components to version 1009, but also all changes lead to a fail and arclog files start multiplying and citadel no longer starts.
The problem is still not solved. I tried it on another test server x86 Debian, as soon as I change any settings for the user, citadel crashes and a large number of log.xxxx files start to be created in the /usr/local/citadel/data folder and the service immediately crashes.
Please enable core dumps on your system. (If you don't know how to do that, check your favorite search engine.)
After the Citadel Server crashes, you can then do this:
cd /usr/local/citadel
gdb ./citserver [path/to/core_file]
The debugger will start up and put you into the server program right after it crashed. Now do this:
thread apply all bt
And post the output of that command. It is also imperative that you are running the latest version. Your problem sounds like one we fixed a couple of versions ago where a corrupt incoming message (usually a spam) causes the server to crash whenever the indexer is run.
I've just compiled the current version of citadel (1009) from source.
It looks like the options for citserver have changed from the documentation. On https://citadel.org/citserver.html there is a -d option to run as a daemon, but on the version I've compiled there is no -d option but there is a -r option I can't find documented anywhere. Are there more up to date docs or have I missed a compile option?
On the same vein I can't find a sample init script anywhere. I'm trying to install on Slackware. Happy to create my own init script but it would be good to have either a sysvinit or systemd service file as an example. Are there any examples around?
I'm trying to follow the setup instructions at https://citadel.org/system_administration_manual.html. It mentions logging in with the citadel binary but when I compiled there was no citadel binary built. I note it is not referenced at https://citadel.org/documentation.html. Is this now deprecated in favor of webciv?
Thanks in advance for any pointers :)
Unfortunately there is no longer a `-d` option. citserver now always runs "in the foreground" because most sites are running it under systemd, which waits for the process to exit so that some action can be taken. If you do need to run citserver from some other init system that requires it to go into the background, simply put an `&` after the citserver command.
Thanks for that. Do you know what the '-r' option to citserver is for?
Also should the citadel binary have been built? When I compile 1009 I'm not getting one.
how do i connect anetwork room with another citadel bbs?
Thanks for that. Do you know what the '-r' option to citserver is for?
Also should the citadel binary have been built? When I compile 1009 I'm not getting one.
'-r' is an undocumented switch that tells Citadel Server not to drop root permissions after initializing. There is normally no reason to do this unless you're troubleshooting a special condition.
The 'citadel' binary is the text mode client. If you are manually compiling the components of the Citadel system, you'll find that build in the textclient/ directory.
how do i connect anetwork room with another citadel bbs?
Room sharing was discontinued a long time ago. It will likely reappear in some other form in the future, probably some sort of fediverse type of thing where we can integrate with more than just other Citadels.
Do you have two or more Citadel sites that you need to string together with room sharing? You might be able to bodge something together by having them pull each other's RSS feeds for each shared room. I haven't tried it but it might just work.
Please enable core dumps on your system. (If you don't know how to do that, check your favorite search engine.)
After the Citadel Server crashes, you can then do this:
cd /usr/local/citadel
gdb ./citserver [path/to/core_file]
The debugger will start up and put you into the server program right after it crashed. Now do this:
thread apply all bt
And post the output of that command. It is also imperative that you are running the latest version. Your problem sounds like one we fixed a couple of versions ago where a corrupt incoming message (usually a spam) causes the server to crash whenever the indexer is run.
Hello!
Here is my core dump log:
root@mail:/usr/local/citadel# coredumpctl debug
PID: 570622 (citserver)
UID: 1001 (citadel)
GID: 1003 (citadel)
Signal: 6 (ABRT)
Timestamp: Fri 2025-03-07 09:08:38 MSK (3min 29s ago)
Command Line: /usr/local/citadel/citserver
Executable: /usr/local/citadel/citserver
Control Group: /system.slice/citadel.service
Unit: citadel.service
Slice: system.slice
Boot ID: 70b64ff8afd2457181058ad8f9bb4a71
Machine ID: 3b023f1e7c6d4965a3d0e2f737442736
Hostname: mail.vodolaga.ru
Storage: /var/lib/systemd/coredump/core.citserver.1001.70b64ff8afd2457181058ad8f9bb4a71.570622.1741327718000000.zst (present)
Size on Disk: 542.9K
Message: Process 570622 (citserver) of user 1001 dumped core.
Stack trace of thread 570624:
#0 0x00007fc28c71cebc __pthread_kill_implementation (libc.so.6 + 0x8aebc)
#1 0x00007fc28c6cdfb2 __GI_raise (libc.so.6 + 0x3bfb2)
#2 0x00007fc28c6b8472 __GI_abort (libc.so.6 + 0x26472)
#3 0x00007fc28c711430 __libc_message (libc.so.6 + 0x7f430)
#4 0x00007fc28c72683a malloc_printerr (libc.so.6 + 0x9483a)
#5 0x00007fc28c72871c _int_free (libc.so.6 + 0x9671c)
#6 0x00007fc28c72af1f __GI___libc_free (libc.so.6 + 0x98f1f)
#7 0x000055633ce570ce fixed_output (citserver + 0x230ce)
#8 0x00007fc28ced0b0e mime_decode (libcitadel.so.4 + 0x8b0e)
#9 0x00007fc28ced124f recurseable_mime_parser (libcitadel.so.4 + 0x924f)
#10 0x00007fc28ced0d48 the_mime_parser (libcitadel.so.4 + 0x8d48)
#11 0x00007fc28ced14c6 mime_parser (libcitadel.so.4 + 0x94c6)
#12 0x000055633ce5948f CtdlOutputPreLoadedMsg (citserver + 0x2548f)
#13 0x000055633ce7d9ff ft_index_message (citserver + 0x499ff)
#14 0x000055633ce7e088 do_fulltext_indexing (citserver + 0x4a088)
#15 0x000055633ce621b5 PerformSessionHooks (citserver + 0x2e1b5)
#16 0x000055633ce4d28d do_housekeeping (citserver + 0x1928d)
#17 0x000055633ce64648 worker_thread (citserver + 0x30648)
#18 0x00007fc28c71b1c4 start_thread (libc.so.6 + 0x891c4)
#19 0x00007fc28c79b85c __clone3 (libc.so.6 + 0x10985c)
Stack trace of thread 570622:
#0 0x00007fc28c761505 __GI___clock_nanosleep (libc.so.6 + 0xcf505)
#1 0x00007fc28c765e13 __GI___nanosleep (libc.so.6 + 0xd3e13)
#2 0x00007fc28c791205 usleep (libc.so.6 + 0xff205)
#3 0x000055633ce64b43 go_threading (citserver + 0x30b43)
#4 0x000055633ce61519 main (citserver + 0x2d519)
#5 0x00007fc28c6b924a __libc_start_call_main (libc.so.6 + 0x2724a)
#6 0x00007fc28c6b9305 __libc_start_main_impl (libc.so.6 + 0x27305)
#7 0x000055633ce45e51 _start (citserver + 0x11e51)
Stack trace of thread 570623:
#0 0x00007fc28c79095c __GI___select (libc.so.6 + 0xfe95c)
#1 0x000055633ce64161 worker_thread (citserver + 0x30161)
#2 0x00007fc28c71b1c4 start_thread (libc.so.6 + 0x891c4)
#3 0x00007fc28c79b85c __clone3 (libc.so.6 + 0x10985c)
ELF object binary architecture: AMD x86-64
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/local/citadel/citserver...
[New LWP 570624]
[New LWP 570622]
[New LWP 570623]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/citadel/citserver'.
Program terminated with signal SIGABRT, Aborted.
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
44 ./nptl/pthread_kill.c: No such file or directory.
[Current thread is 1 (Thread 0x7fc288b8c6c0 (LWP 570624))]
(gdb) thread apply all bt
Thread 3 (Thread 0x7fc28938d6c0 (LWP 570623)):
#0 0x00007fc28c79095c in __GI___select (nfds=39, readfds=0x7fc28938cbc0, writefds=0x0, exceptfds=0x0, timeout=0x7fc28938cbb0) at ../sysdeps/unix/sysv/linux/select.c:69
#1 0x000055633ce64161 in worker_thread (blah=0x0) at server/sysdep.c:811
#2 0x00007fc28c71b1c4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#3 0x00007fc28c79b85c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Thread 2 (Thread 0x7fc289c1a980 (LWP 570622)):
#0 0x00007fc28c761505 in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=req@entry=0x7ffe48735c60, rem=rem@entry=0x0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:48
#1 0x00007fc28c765e13 in __GI___nanosleep (req=req@entry=0x7ffe48735c60, rem=rem@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:25
#2 0x00007fc28c791205 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:31
#3 0x000055633ce64b43 in go_threading () at server/threads.c:137
#4 0x000055633ce61519 in main (argc=1, argv=0x7ffe48736e68) at server/server_main.c:297
Thread 1 (Thread 0x7fc288b8c6c0 (LWP 570624)):
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1 0x00007fc28c71cf1f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2 0x00007fc28c6cdfb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3 0x00007fc28c6b8472 in __GI_abort () at ./stdlib/abort.c:79
#4 0x00007fc28c711430 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fc28c82b459 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#5 0x00007fc28c72683a in malloc_printerr (str=str@entry=0x7fc28c82e180 "free(): invalid next size (normal)") at ./malloc/malloc.c:5660
#6 0x00007fc28c72871c in _int_free (av=0x7fc284000030, p=0x7fc284026a30, have_lock=<optimized out>, have_lock@entry=0) at ./malloc/malloc.c:4592
#7 0x00007fc28c72af1f in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3385
#8 0x000055633ce570ce in fixed_output (name=0x7fc2840344e0 "", filename=0x7fc2840364f0 "", partnum=0x7fc28cee3103 "1", disp=0x7fc2840374f8 "", content=0x7fc284039520, cbtype=0x7fc2840314c8 "text/html", cbcharset=0x7fc28
40324d0 "utf-8", length=42916, encoding=0x7fc288b892a0 "binary", cbid=0x7fc284038500 "", cbuserdata=0x7fc288b8b990) at server/msgbase.c:1177
#9 0x00007fc28ced0b0e in mime_decode (partnum=0x7fc28cee3622 "", part_start=0x7fc284015f39 "\n<!DOCTYPE html=0A PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http:=\n//www.w3.org/TR/xhtml1/DTD/xhtml1-transition
al.dtd\">=0A<html xmlns=3D\"http:=\n//www.w3.org/1999/xhtml\">=0A=0A<head>=0A <"..., length=<optimized out>, content_type=0x7fc2840314c8 "text/html", charset=0x7fc2840324d0 "utf-8", encoding=<optimized out>, dispositi
on=0x7fc2840374f8 "", id=0x7fc284038500 "", name=0x7fc2840344e0 "", filename=0x7fc2840364f0 "", CallBack=0x55633ce56f00 <fixed_output>, PreMultiPartCallBack=0x0, PostMultiPartCallBack=0x0, userdata=0x7fc288b8b990, dont_d
ecode=0) at lib/mime_parser.c:238
#10 0x00007fc28ced124f in recurseable_mime_parser (partnum=partnum@entry=0x7fc28cee3622 "", content_start=0x7fc284015f39 "\n<!DOCTYPE html=0A PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http:=\n//www.w3.org/TR
/xhtml1/DTD/xhtml1-transitional.dtd\">=0A<html xmlns=3D\"http:=\n//www.w3.org/1999/xhtml\">=0A=0A<head>=0A <"..., content_end=content_end@entry=0x7fc284022933 "", CallBack=CallBack@entry=0x55633ce56f00 <fixed_output>,
PreMultiPartCallBack=PreMultiPartCallBack@entry=0x55633ce56dac <fixed_output_pre>, PostMultiPartCallBack=PostMultiPartCallBack@entry=0x55633ce56e56 <fixed_output_post>, userdata=0x7fc288b8b990, dont_decode=0, m=0x7fc284
02e4b0) at lib/mime_parser.c:712
#11 0x00007fc28ced0d48 in the_mime_parser (partnum=partnum@entry=0x7fc28cee3622 "", content_start=<optimized out>, content_end=0x7fc284022933 "", CallBack=0x55633ce56f00 <fixed_output>, PreMultiPartCallBack=0x55633ce56da
c <fixed_output_pre>, PostMultiPartCallBack=0x55633ce56e56 <fixed_output_post>, userdata=0x7fc288b8b990, dont_decode=0) at lib/mime_parser.c:811
#12 0x00007fc28ced14c6 in mime_parser (content_start=<optimized out>, content_end=<optimized out>, CallBack=<optimized out>, PreMultiPartCallBack=<optimized out>, PostMultiPartCallBack=<optimized out>, userdata=<optimize
d out>, dont_decode=0) at lib/mime_parser.c:836
#13 0x000055633ce5948f in CtdlOutputPreLoadedMsg (TheMessage=0x7fc284025a20, mode=0, headers_only=0, do_proto=0, crlf=1, flags=0) at server/msgbase.c:2074
#14 0x000055633ce7d9ff in ft_index_message (msgnum=14539, op=1) at server/modules/fulltext/serv_fulltext.c:86
#15 0x000055633ce7e088 in do_fulltext_indexing () at server/modules/fulltext/serv_fulltext.c:242
#16 0x000055633ce621b5 in PerformSessionHooks (EventType=12) at server/serv_extensions.c:439
#17 0x000055633ce4d28d in do_housekeeping () at server/housekeeping.c:127
#18 0x000055633ce64648 in worker_thread (blah=0x0) at server/sysdep.c:951
#19 0x00007fc28c71b1c4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x00007fc28c79b85c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb)
Interesting, I didn't know there was something called "coredumpctl". I've always had to describe the tedious process of finding the correct core dump and feeding it into the debugger. Once again, systemd has made our lives easier.
And from what I can tell from a brief glance at your stack trace, it looks like some really malformed message is crashing the MIME parser. We'll dig into that and see if we can figure out what conditions cause that. If you can manage to save a copy of the original RFC822 message that'd do it quickly, but we'll find it one way or another.
If you can manage to save a copy of the original RFC822 message that'd do it quickly, but we'll find it one way or another.
How to save this message from DB?
Also should the citadel binary have been built? When I compile 1009 I'm not getting one.
The 'citadel' binary is the text mode client. If you are manually compiling the components of the Citadel system, you'll find that build in the textclient/ directory.
I've got no textclient directory, nothing in the build directory called 'citadel'.
What ist best pratice for journaling mails ? Wich Mail Storage is useful ?
Mailstore or other programs who can handle journaling mails from citadel ?
Cheers
Dear Maintainers,
I am being under attack since for two days now.
I start citadel and some time after appears .logs files in the data directory until disk full and citadel become unusable.
Please give me a solution. This way citadel become unusable.
What you need? The log files?
Thanks,
Luís.
Not that it helps you any, but that is one of the main reasons i shut down all my public facing sites and now even my personal stuff has to go thru a password protected proxy, THEN a 2nd password in the app.
I let the proxy deal with the flood instead of my actual apps. Safer that way.
For a while it was pulled back to vpn only.. but the flood subsided enough to do the proxy instead.
Dear Maintainers,
I am being under attack since for two days now.
I start citadel and some time after appears .logs files in the data directory until disk full and citadel become unusable.
Please give me a solution. This way citadel become unusable.
What you need? The log files?
Thanks,
Luís.
How to save this message from DB?
If you are able to get your Citadel server running for a little while, you can just do
cd /usr/local/citadel
./sendcommand MSG2 14539
But if you can't, you might try extracting it from the database:
cd /usr/local/citadel
./ctdldump -y | grep 'msgtext|14539'
What ist best pratice for journaling mails ? Wich Mail Storage
is useful ?
Citadel doesn't define this, it just creates a mailbox that collects the journals.
Usually when someone wants to implement this function, they use a service that collects the journals using POP3. But there's no specific requirement.