Installed docker.io on OrangePi 5 Plus running debian.  Followed the instructions to get a "permanent" running citadel instance.  What is the default admin login for the container or is it the first user created?

Found it - admin/citadel.  Changed for my installation.  thanks

I tried the easy install because I have always loved Citadel but when it's compiling, I end up getting an error for an undefined reference to libiconv and libiconv-close in libcitadel. Anyone else have this problem?

Hi,

I am running a fully patched release of raspian bullseye on a raspberry pi. Installing Citadel using the Easy Install method.

Having done all the compiles and answered all the configuration questions,  Citadel restarts then issues the following message:

Restarting Citadel server to apply changes

bash: line 643: 26257 Segmentation fault      /usr/local/citadel/setup < /dev/tty > /dev/tty 2> /dev/tty

 Citadel Easy Install is aborting. 

 The last few lines above this message may indicate what went wrong. 

Here is a good one....

Last Wednesday, just as end of day, the Cisco ASA firewall went down on the NAT for Citadel..... version 972

While there are automated backups running everyday, everything continued to work..... the backup MX took over spooling email traffic, and the Citadel continued perating within the environment, no mail was flowing due to the disconnected NAT.

On Friday, the IP of the OUTSIDE Citadel got listed at Spamhaus ZEN.

Then the later Friday, the AD LDAP connection failed.

Backups continued, however, the database was encountering problems which was corruption. 

Monday morning, client called NO emails.

Found the AD down, and was unable to connect to AD server. Citadel was unresponsive. ran sysemctl stop on Citadel apps (incl webcit) and restarted. AD restored.

Found the firewall was not NATted. Reset the the firewall (full power reset as ASA wasnt reachable)

Mail started to flow, but AD failed again causing 551 user unknown, emails rejected.

De-spooled MX backup server sent emails which were rejected (3.5GB)

Mail from Friday to Monday lost.

Got Citadel up again with AD, but seems the database was corrupted. Ran db_recovery, no joy. Ran database_cleanup, mail outbound worked, inbound appeared to work as headers came through, but no body. Attempted to restore from backups going back each day from Monday. Several cdb failed db_verify.

So all the way back to Wednesday were restored. All clients were were backed up to PST files. And tried to update to 996. System easyinstall ran fine.....until the setup portion of the install where it hung. No activity for several hours and no joy.

Im restoring the backup of the 972 as I cant wait to figure out where issues are as I must create new cdb and load all the PSTs to return ops.

I cant figure out where it went wrong.....

Well,

A number of users are attempting to clean their email accounts of Deleted Messages and Trash and the citadel user "citadel" user account 1001 is seeing application crashes:

This system has been functional since last year with nary an issue. 956 was initial system. Currently at 972.

It appears that webcit-http did a core dump yesterday with an issue in libcitadel. I could provide log snippets on request. The core dumps today appear related to the imap-cleanup procedure. A number of users are clearing the Deleted Messages and Trash in their accounts as well as Main Floor Trashcan. We have had two core dumps in as many hours. A couple of users have more than 10K emails to empty....

I

Running 972 on a Redhat 9 EL

LDAP Microsoft AD on 2008R2 Forest/Domain

Aide has created 820 MILLION accounts

User Creation Notice

New user account <> has been created, from host []

Happens at various times periods, anywhere from 1 to 6 per minutes.

I suspect our friends at Microsoft as the culprit as several other Citadel server dont exhibit this behavior. There are several AD authentication schemas but not at the version with the spurious  account creations

Anyone see this before.

And yes the plan is to go to 996 as rapidly as possible........

Cheers

I can send some logs and configs from the misbehaving system.

I also would like to maillog to show the local recipient. Is there a way to expand the level of the logging to show this?

Just an observation i have been meaning to mention for a while and its nothing critical at all:

When a room description is larger than can display, and you hover over, the full text popup normally only displays for a second or so, then the popup is gone. ( at least for me. Chrome. Several versions. Linux )

The DB xxxx12345.log files in "data" have dramically increased in number, and are creating new ones by 10+ every second.

The citadel application core dumps on "systemctl stop citadel" when executing a backup.

I this a concern or and induication that something is misconfigured? All Citadel related files are owned citadel and group citadel.

OK, This log.xxxxxx process count goes to about 1600 log files then they are processed and committed (?) so I am thinking this is by design making the 996 DB more resilient. I need to look into the core dump issue on stopping Citadel......

Fri Dec 15 2023 14:09:05 EST from stormytramp Subject: V 996 The DB Log count has increased drammitcally

The DB xxxx12345.log files in "data" have dramically increased in number, and are creating new ones by 10+ every second.

 

The citadel application core dumps on "systemctl stop citadel" when executing a backup.

 

I this a concern or and induication that something is misconfigured? All Citadel related files are owned citadel and group citadel.

The DB xxxx12345.log files in "data" have dramically increased in number, and are creating new ones by 10+ every second.

This suggests that something is very rapidly writing to your database.

Check to make sure your site is not under attack by spammers, and that you don't have any runaway scripts running, anything like that.

And this might be a long shot, but by chance did you run the load test utility and forgot to stop it?

The citadel application core dumps on "systemctl stop citadel" when executing a backup.

That normally doesn't happen anymore, but if you're able to produce a core dump, run it through the debugger and let's see where it crashed.

OK, This log.xxxxxx process count goes to about 1600 log files then they are processed and committed (?) so I am thinking this is by design making the 996 DB more resilient. 

Well, sort of.  :)

The fact that it's committing 16 GB at a time and not crashing is the new and improved database.  The fact that you are easily recovering from that without corruption when you do take it down is also the new and improved database.  But the fact that it's happening is indicative of unwanted server activity.

You could also run one of those log files through the `strings` utility to see what's inside them.

Well, this is embarrassing...

So, I was (thinking I was) cleaning up the issues from the VM crash, inadvertently copied over the DB & log files over the running server DB files and crashed the system.....

Once I restored from the backup, those log files are no longer being created. I didnt run the load test, so I can only assume that I missed something during the upgrade. I also found the the webcit systemd files were not created, so I took care of that. Again, I don't think it was the easy install procedure because I created a new server to check out the process. So all looks as it should.

Another note, regarding the mail logging facility, I have been able to get the logs to reflect all the pertinent data looking for in rsyslog (Redhat 9)... I did not modify the citserver systemd files, but the rsyslog file. Am I doing the wrong files for the details Im looking for?

Cheers

Sat Dec 16 2023 18:41:02 EST from IGnatius T Foobar Subject: Re: V 996 The DB Log count has increased drammitcally

OK, This log.xxxxxx process count goes to about 1600 log files then they are processed and committed (?) so I am thinking this is by design making the 996 DB more resilient. 

Well, sort of.  :)

The fact that it's committing 16 GB at a time and not crashing is the new and improved database.  The fact that you are easily recovering from that without corruption when you do take it down is also the new and improved database.  But the fact that it's happening is indicative of unwanted server activity.

You could also run one of those log files through the `strings` utility to see what's inside them.