Language:
switch to room list switch to menu My folders
Go to page: First ... 20 21 22 23 [24] 25 26 27 28 ... Last
[#] Wed Oct 21 2020 08:48:40 EDT from platonov @ Uncensored

Subject: Re: Using c_shrink_db_files to compact the database..

[Reply] [ReplyQuoted] [Headers] [Print]

Unfortunately when you do any of those ./sendcommand "CONF GETVAL|c_shrink_db_files" on Cit 8.24 you get an error msg:

./sendcommand "CONF GETVAL|c_shrink_db_files"
sendcommand: started (pid=18230) connecting to Citadel server at /usr/local/citadel/citadel-admin.socket
200 antimatrix Citadel server ADMIN CONNECTION ready.
CONF GETVAL|c_shrink_db_files
512 Illegal option(s) specified.
sendcommand: processing ended.

./sendcommand "CONF LISTVAL"
sendcommand: started (pid=18232) connecting to Citadel server at /usr/local/citadel/citadel-admin.socket
200 antimatrix Citadel server ADMIN CONNECTION ready.
CONF LISTVAL
512 Illegal option(s) specified.

Tue Oct 20 2020 17:17:57 EDT from platonov @ Uncensored Subject: Re: Using c_shrink_db_files to compact the database..

Thanx a lot for your info and efforts. Appreciated indeed.

Tue Oct 20 2020 13:45:05 EDT from warbaby @ Uncensored Subject: Using c_shrink_db_files to compact the database..

Doing some testing.. You'll definitely want to put quotes around the command.

/usr/local/citadel/sendcommand "CONF PUTVAL|c_shrink_db_files|1"

It does not show up with..

./sendcommand "CONF GET"

To read the value that's in there..

./sendcommand "CONF GETVAL|c_shrink_db_files"

Also

./sendcommand "CONF LISTVAL"

or 

./sendcommand "CONF LISTVAL" | grep shrink

I'll see if I can trigger a nightly cleanup on my dev box and let everyone know how it goes..

 


 



 



[#] Wed Oct 21 2020 09:19:01 EDT from platonov @ Uncensored

Subject: Re: Is there a problem of incorrect displaying of Subject: header?

[Reply] [ReplyQuoted] [Headers] [Print]

Well, what I have noticed is that the Subject: headers get chopped off not because of size issue, but because of non-ASCII delimeter characters, such as backquotes long dashes and non-ASCII character in general. In combination with the fact that in the migrated version of the database to 9.29 node ALL Subject: headers worked correctly, I wonder if this has something to do with some configurable parameter. The program code is the same. So, it is probably not in the source code, but in data somewhere...

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



[#] Wed Oct 21 2020 09:36:45 EDT from platonov @ Uncensored

Subject: Re: Is there a problem of incorrect displaying of Subject: header?

[Reply] [ReplyQuoted] [Headers] [Print]

I just remembered something that might be significant as far as *** Citadel migration was unsuccessful. *** msg.

(We are talking about Cit v. 8.24 here)

I have one bad room that does not have a room name and when you click on it it looks like a single blank character as name. So, I wonder if this could be the reason the migration process bombs out after all rooms have been migrated OK, and, secondly, is there a way to delete that room? May be moving all the other rooms on that floor to some other floor and the deleting the entire floor?

Wed Oct 21 2020 08:41:59 EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



[#] Wed Oct 21 2020 11:47:49 EDT from platonov @ Uncensored

Subject: Re: Is there a problem of incorrect displaying of Subject: header?

[Reply] [ReplyQuoted] [Headers] [Print]

 

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

As far as "It's got to be a buffer size issue", I doubt that one. Because the Subject: header gets chopped off when there is non-ASCII character, delimiter, etc. Or it may have to do something with character sets or some special processing of Subject: header if there is any.

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And the kicker is, the Subject: headers were magically working correctly. Not a single one was trimmed down. I did verify quite a bit of records and everything looked just fine. Does it mean that truncation of Subject header is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

root@12.34.56.78:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

But database did transfer OK, except when you try to restart citserver you get the error status:

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ago
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, restart counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too quickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-code'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

What can be done to fix the issue of not being able to start citserver and keep the migrated data? I was thinking of doing a database recover kind of thing. Is there a chance that it gets fixed?

As far as *** Citadel migration was unsuccessful. *** msg.

(We are talking about Citadel 8.24 here)

I have one room that does not have a room name and when you click on it on the desktop it looks like a single blank character as a name. So, I wonder if this could be the reason the migration process bombs out after all rooms have been migrated OK.

Secondly, is there a way to delete that room? May be moving all the other rooms on that floor to some other floor and the deleting the entire floor?




[#] Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored

Subject: Re: Is there a problem of incorrect displaying of Subject: header?

[Reply] [ReplyQuoted] [Headers] [Print]

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



[#] Wed Oct 21 2020 13:41:21 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate fails / citserver does not restart properly afterwords

[Reply] [ReplyQuoted] [Headers] [Print]

I am going to verify that ctdlmigrate bombs out because of these key files. I'll just move them temporarily out of /usr/local/citadel path and try to migrate again.

I'd like to know why citserver does not start properly if this is an issue with these key files. I am not worried about whether migration is not totally complete as I only need the messages that I know did migrate OK.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@12.34.56.78:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ago
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=105)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, restart counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too quickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-code'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



[#] Wed Oct 21 2020 13:52:26 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate and Citadel Key files

[Reply] [ReplyQuoted] [Headers] [Print]

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



[#] Wed Oct 21 2020 14:38:53 EDT from warbaby @ Uncensored

Subject: Re: Ctdlmigrate and Citadel Key files

[Reply] [ReplyQuoted] [Headers] [Print]

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



[#] Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored

Subject: Re: Ctdlmigrate and Citadel Key files

[Reply] [ReplyQuoted] [Headers] [Print]

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

 

Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



[#] Wed Oct 21 2020 15:24:31 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate and Citadel Key files

[Reply] [ReplyQuoted] [Headers] [Print]

I tried to move out the Citadel keys from /usr/local/citadel/keys to /usr/local/citadel/keys/bak and then rerun ctdlmigrate. This is the output of it now:

rsync -va --rsh='ssh -S /tmp/ctdl.5fbe.579478fe' root@12.34.56.78:/usr/local/citadel/messages// /usr/local/citadel/messages//
receiving incremental file list
./
aideopt
changepw
dotopt
entermsg
entopt
goodbye
hello
help
mainmenu
newuser
readopt
register
roomaccess
unlisted

sent 293 bytes  received 6,154 bytes  12,894.00 bytes/sec
total size is 5,328  speedup is 0.83
false
rsync -va --rsh='ssh -S /tmp/ctdl.5fbe.579478fe' root@12.34.56.78:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list
./
citadel.cer
citadel.key

sent 95 bytes  received 5,464 bytes  11,118.00 bytes/sec
total size is 5,286  speedup is 0.95
false
false

 *** Citadel migration was unsuccessful. ***

Shutting down the socket connection...

Exit request sent.

===============================================================
Moving out the Keys files did not help.
We still get the same error.

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2020-10-21 18:17:03 UTC; 2min 44s ago
    Process: 24630 ExecStart=/usr/local/citadel/citserver (code=exited, status=105)
   Main PID: 24630 (code=exited, status=105)

Oct 21 18:17:03 UbuToo systemd[1]: citadel.service: Scheduled restart job, restart counter is at 6.
Oct 21 18:17:03 UbuToo systemd[1]: Stopped Citadel Server.
Oct 21 18:17:03 UbuToo systemd[1]: citadel.service: Start request repeated too quickly.
Oct 21 18:17:03 UbuToo systemd[1]: citadel.service: Failed with result 'exit-code'.
Oct 21 18:17:03 UbuToo systemd[1]: Failed to start Citadel Server.

I wonder what those status msgs mean, like "Start request repeated too quickly" and "Scheduled restart job, restart counter is at 6".

From what it looks to me is some kind of a database damage, or maybe some incompatibility creeping through between cit 8.24 and 9.29 versions.

Do you think it makes sense to try to recover the database?

Basically, when I restore the original database files Citadel runs just fine. But when I use the files produced after migration then we get the citadel start problem.

What would be the "right" course of action here? Looks like there is no way to compact the 8.24 database and it is over 50 Gigs now and about thousands of times bigger than actual data records are. So, about the only option I have is to make the migrated version work, which is beginning to look pretty nasty.

Wed Oct 21 2020 14:38:53 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



[#] Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate and Citadel Key files

[Reply] [ReplyQuoted] [Headers] [Print]

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



[#] Wed Oct 21 2020 16:15:10 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate and Citadel Key files

[Reply] [ReplyQuoted] [Headers] [Print]

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



[#] Wed Oct 21 2020 17:28:50 EDT from warbaby @ Uncensored

Subject: Re: Ctdlmigrate

[Reply] [ReplyQuoted] [Headers] [Print]

I would put the effort into your source database files first..

A). Just because you can't set c_db_shrink_files locally, doesn't mean BerkleyDB compact is out of reach.

The DB->compact() method can be reproduced with a fairly trivial c program. I am looking at some other utility packages to see if someone already has written it..

B). "slacker method".. Drag/drop messages over IMAP. May not be terrible if both boxes are local to each other. I have done many systems this way.. admittedly nothing like 50G!

C).  You may have the 'migrate' utility with your original version.  That's an XML exporter.  They at least you'd have a parse-able archive.

Wed Oct 21 2020 04:15:10 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



 



[#] Thu Oct 22 2020 12:27:06 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate

[Reply] [ReplyQuoted] [Headers] [Print]

I'll insert my stuff in-place of your text.

Wed Oct 21 2020 17:28:50 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate

I would put the effort into your source database files first..

A). Just because you can't set c_db_shrink_files locally, doesn't mean BerkleyDB compact is out of reach.

The DB->compact() method can be reproduced with a fairly trivial c program. I am looking at some other utility packages to see if someone already has written it..

That would be great. I'd LOVE to run something to reduce database size by factor of 10000.
I happen to have citadel 8.24 source tree and was thinking of may be just adding the DB->compact call after each purge, even without using some test variable.
Can you tell me what do I need to do to build citadel?

B). "slacker method".. Drag/drop messages over IMAP. May not be terrible if both boxes are local to each other. I have done many systems this way.. admittedly nothing like 50G!

I am not quite following this "Drag/drop messages over IMAP". What exactly do I have to do? My 2 boxes are on the same lan, but it is at virtual server provider facilities.

C).  You may have the 'migrate' utility with your original version.  That's an XML exporter.  They at least you'd have a parse-able archive.

Unfortunately, I don't seem to have the 'migrate' utility, at least in /usr/local/ctdlsupport or /usr/local/citadel dirs. But I vaguely recall something about the XML export/import functionality of cit 824. Do you happen to know where I can find some info on it, if there is any? Looks like old docs, where it is described, are gone, and the new ones do not seem to mention it.

Wed Oct 21 2020 04:15:10 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



 



 



[#] Thu Oct 22 2020 12:58:06 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate

[Reply] [ReplyQuoted] [Headers] [Print]

To warbaby,

I want to try to add cdb_compact() call on every purge procedure in Citadel and see what happens as far as database compaction goes.

But my source is cit 8.24 and yours is different. You say:

Right at line 870 of citadel/modules/expire/serv_expire.c we find..

if ( (!server_shutting_down) && (CtdlGetConfigInt("c_shrink_db_files") != 0) )
{
    cdb_compact();             // Shrink the DB files on disk
}

Could you give me a few surrounding lines or just tell me after what and before what it should be done, so that I plug-in that call in the correct place?

Thu Oct 22 2020 12:27:06 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate

I'll insert my stuff in-place of your text.

Wed Oct 21 2020 17:28:50 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate

I would put the effort into your source database files first..

A). Just because you can't set c_db_shrink_files locally, doesn't mean BerkleyDB compact is out of reach.

The DB->compact() method can be reproduced with a fairly trivial c program. I am looking at some other utility packages to see if someone already has written it..

That would be great. I'd LOVE to run something to reduce database size by factor of 10000.
I happen to have citadel 8.24 source tree and was thinking of may be just adding the DB->compact call after each purge, even without using some test variable.
Can you tell me what do I need to do to build citadel?

B). "slacker method".. Drag/drop messages over IMAP. May not be terrible if both boxes are local to each other. I have done many systems this way.. admittedly nothing like 50G!

I am not quite following this "Drag/drop messages over IMAP". What exactly do I have to do? My 2 boxes are on the same lan, but it is at virtual server provider facilities.

C).  You may have the 'migrate' utility with your original version.  That's an XML exporter.  They at least you'd have a parse-able archive.

Unfortunately, I don't seem to have the 'migrate' utility, at least in /usr/local/ctdlsupport or /usr/local/citadel dirs. But I vaguely recall something about the XML export/import functionality of cit 824. Do you happen to know where I can find some info on it, if there is any? Looks like old docs, where it is described, are gone, and the new ones do not seem to mention it.

Wed Oct 21 2020 04:15:10 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



 



 



 



[#] Thu Oct 22 2020 13:15:14 EDT from warbaby @ Uncensored

Subject: Re: Ctdlmigrate "Slacker Method"

[Reply] [ReplyQuoted] [Headers] [Print]

By "Slacker Method" I mean:

Setting up two imap accounts., such as in Thunderbird or another email client.

[me@old-server.org]

  INBOX

  DRAFTS

  TRASH

 ... etc..

[me@new-server.net]

   INBOX

   DRAFTS

  TRASH

  etc..

[a very minimal example, supposed to look vaguely like account folders in Thunderbird..]

1) Select, highlight all messages in the preview pane of the old server.. [Edit..Select...All]

2) Drag all messages to the new target folder.  

[You might also consider doing this with "Local Folders" bypassing the new box at all. but then you'll have to copy them back.. .]

I've done this successfully many times.  You can also put out instructions to have your users migrate their own.

Afterward, just make the dns change and "new-server" becomes "orig-server"] ..

I completely understanding this is going to be case dependent and won't work for everybody.. BUT .. it does actually work. 

Pretty easy to scrape old room/list archives this way..if you are patient, and methodical.

Also consider the use of Add-ons like 'Remove Duplicate Messages".. that might be helpful for some of you.

I use that one quite a bit.

Also be aware! MESSAGES may be LOST if the POWER GOES OUT while copying/moving!

Or due to a network outage.. so use with caution/intelligence.

 

Thu Oct 22 2020 12:27:06 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate

I'll insert my stuff in-place of your text.

Wed Oct 21 2020 17:28:50 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate

I would put the effort into your source database files first..

A). Just because you can't set c_db_shrink_files locally, doesn't mean BerkleyDB compact is out of reach.

The DB->compact() method can be reproduced with a fairly trivial c program. I am looking at some other utility packages to see if someone already has written it..

That would be great. I'd LOVE to run something to reduce database size by factor of 10000.
I happen to have citadel 8.24 source tree and was thinking of may be just adding the DB->compact call after each purge, even without using some test variable.
Can you tell me what do I need to do to build citadel?

B). "slacker method".. Drag/drop messages over IMAP. May not be terrible if both boxes are local to each other. I have done many systems this way.. admittedly nothing like 50G!

I am not quite following this "Drag/drop messages over IMAP". What exactly do I have to do? My 2 boxes are on the same lan, but it is at virtual server provider facilities.

C).  You may have the 'migrate' utility with your original version.  That's an XML exporter.  They at least you'd have a parse-able archive.

Unfortunately, I don't seem to have the 'migrate' utility, at least in /usr/local/ctdlsupport or /usr/local/citadel dirs. But I vaguely recall something about the XML export/import functionality of cit 824. Do you happen to know where I can find some info on it, if there is any? Looks like old docs, where it is described, are gone, and the new ones do not seem to mention it.

Wed Oct 21 2020 04:15:10 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



 



 



 



[#] Thu Oct 22 2020 14:14:51 EDT from warbaby @ Uncensored

Subject: Rolling our own DB->compact() binary..

[Reply] [ReplyQuoted] [Headers] [Print]

There are a lot of things that come to mind about this.. namely

* This would be a great utility for all of us, and everything we need to create it is within our reach.

* We should do it!  Collaborate! [Somebody out there probably has time and ambition to finish this today.. ]

* I'm not the greatest C programmer in the world.  I've written more Pascal than C.. and that, quite a while ago..also, I have a lot of other things going on.. so feel free to test your skills.. :)

* Even so, it's is merely a minor academic exercise. It's filename, a method, and a return value.. Not exactly requiring Linus Torvalds.  I will probably get to it sometime..

* Berkley versions/idiosyncrasies may be different from Citadel 8.x to 9.x, it should be researched.

* All the code to which show the use of compact() are in the citadel repo (I will attach the main files dealing with this).

* These methods may not exist in the older code. (check your database.c)

* It would be nice to have it a stand-alone distributable program, w/code & binaries..  using only the necessary bdb libraries.. not dragging all or most of the citadel code along with it..

* FYI, I'm running Debian 10...

* The binary should take one file name as input, and compact that file, returning any errors, possibly any interesting statistics.

* Iteration through files can be done with a bash script.

* Db->compact() is documented here.

* I have attached server_expire.c (from modules/expire) which shows where cdb_compact() is called during the auto-purger run.

* I have attached database.c which shows how compact() is used.  It can be done with essentially in one line.

* A simple way to get the source which is ready to build..

* wget -q http://easyinstall.citadel.org/install -O easyinstall.sh

* Comment out any lines that "rm -r" .. then run it.. when You're done you'll have a nice source tree..  (or just pull the git repo, but easyinstall is even easier, all sources configured and ready to go..

* If several of us are going to be working on this, and communicating, we might consider moving to another room. 

* Since Citadel Support is actually also a live mailing list.

* It's probably taken me longer to write this than it would take some of you to actually write and compile the program....

So.. "let's do it!"

:)



database.c (text/x-csrc, 30049 bytes) [ View | Download ]
serv_expire.c (text/x-csrc, 31360 bytes) [ View | Download ]
[#] Thu Oct 22 2020 16:56:40 EDT from platonov @ Uncensored

Subject: Re: Ctdlmigrate "Slacker Method"

[Reply] [ReplyQuoted] [Headers] [Print]

I actually tried this method and it worked out pretty nicely. So, now I have my records "ported" from cit 8.24 to 9.29.

Thanx to warbaby for the info.

Thu Oct 22 2020 13:15:14 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate "Slacker Method"

By "Slacker Method" I mean:

Setting up two imap accounts., such as in Thunderbird or another email client.

[me@old-server.org]

  INBOX

  DRAFTS

  TRASH

 ... etc..

[me@new-server.net]

   INBOX

   DRAFTS

  TRASH

  etc..

[a very minimal example, supposed to look vaguely like account folders in Thunderbird..]

1) Select, highlight all messages in the preview pane of the old server.. [Edit..Select...All]

2) Drag all messages to the new target folder.  

Well, you can even drag the entire rooms and drop them on the same floor in destination and it will transfer all the messages.

[You might also consider doing this with "Local Folders" bypassing the new box at all. but then you'll have to copy them back.. .]

I've done this successfully many times.  You can also put out instructions to have your users migrate their own.

Afterward, just make the dns change and "new-server" becomes "orig-server"] ..

I completely understanding this is going to be case dependent and won't work for everybody.. BUT .. it does actually work. 

Pretty easy to scrape old room/list archives this way..if you are patient, and methodical.

Also consider the use of Add-ons like 'Remove Duplicate Messages".. that might be helpful for some of you.

Which program is this add-on for? Email client?

I use that one quite a bit.

Also be aware! MESSAGES may be LOST if the POWER GOES OUT while copying/moving!

Or due to a network outage.. so use with caution/intelligence.

 

Thu Oct 22 2020 12:27:06 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate

I'll insert my stuff in-place of your text.

Wed Oct 21 2020 17:28:50 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate

I would put the effort into your source database files first..

A). Just because you can't set c_db_shrink_files locally, doesn't mean BerkleyDB compact is out of reach.

The DB->compact() method can be reproduced with a fairly trivial c program. I am looking at some other utility packages to see if someone already has written it..

That would be great. I'd LOVE to run something to reduce database size by factor of 10000.
I happen to have citadel 8.24 source tree and was thinking of may be just adding the DB->compact call after each purge, even without using some test variable.
Can you tell me what do I need to do to build citadel?

B). "slacker method".. Drag/drop messages over IMAP. May not be terrible if both boxes are local to each other. I have done many systems this way.. admittedly nothing like 50G!

I am not quite following this "Drag/drop messages over IMAP". What exactly do I have to do? My 2 boxes are on the same lan, but it is at virtual server provider facilities.

C).  You may have the 'migrate' utility with your original version.  That's an XML exporter.  They at least you'd have a parse-able archive.

Unfortunately, I don't seem to have the 'migrate' utility, at least in /usr/local/ctdlsupport or /usr/local/citadel dirs. But I vaguely recall something about the XML export/import functionality of cit 824. Do you happen to know where I can find some info on it, if there is any? Looks like old docs, where it is described, are gone, and the new ones do not seem to mention it.

Wed Oct 21 2020 04:15:10 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



 



 



 



 



[#] Thu Oct 22 2020 17:47:55 EDT from IGnatius T Foobar @ Uncensored

Subject: Re: Is there a problem of incorrect displaying of Subject: header?

[Reply] [ReplyQuoted] [Headers] [Print]

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database

from 8.24 to 9.29 node.

I don't think that's even possible. If it is, then we need to put some code into the migrate utility to prevent that from being done. Citadel has a lot of code that runs when you upgrade, to convert old data formats into new data formats. For those interested, you can look at the source of serv_upgrade.c to see a fascinating timeline of things we've changed over the years and how it handles all of the conversion behind the scenes without bothering you about it.

What it *can't* do is perform that conversion and export/import at the same time.

The correct procedure is to upgrade to the latest version in-place, and *then* migrate it.

I apologize if this isn't clear in the documentation.

[#] Thu Oct 22 2020 20:09:01 EDT from warbaby @ Uncensored

Subject: Re: Ctdlmigrate "Slacker Method"

[Reply] [ReplyQuoted] [Headers] [Print]

I'm really glad to hear it!

Here's the Remove Duplicate Add-on for Thunderbird ..(I'm still using one of the older versions so, should be okay.. )

Also, I usually create the Local Folders

* Sent

* Archives

* Drafts

* Templates (not that I use them, but whatever)

and under 'Account' ... "Settings' .. 'Copies & Folders'

Select those instead of saving everything in Citadel.  Saves a little time/network overhead..

Also, the "Search Button Add-on" .. make 1000% times difference being able to actually find an email..

 

Thu Oct 22 2020 04:56:40 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate "Slacker Method"

I actually tried this method and it worked out pretty nicely. So, now I have my records "ported" from cit 8.24 to 9.29.

Thanx to warbaby for the info.

Thu Oct 22 2020 13:15:14 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate "Slacker Method"

By "Slacker Method" I mean:

Setting up two imap accounts., such as in Thunderbird or another email client.

[me@old-server.org]

  INBOX

  DRAFTS

  TRASH

 ... etc..

[me@new-server.net]

   INBOX

   DRAFTS

  TRASH

  etc..

[a very minimal example, supposed to look vaguely like account folders in Thunderbird..]

1) Select, highlight all messages in the preview pane of the old server.. [Edit..Select...All]

2) Drag all messages to the new target folder.  

Well, you can even drag the entire rooms and drop them on the same floor in destination and it will transfer all the messages.

[You might also consider doing this with "Local Folders" bypassing the new box at all. but then you'll have to copy them back.. .]

I've done this successfully many times.  You can also put out instructions to have your users migrate their own.

Afterward, just make the dns change and "new-server" becomes "orig-server"] ..

I completely understanding this is going to be case dependent and won't work for everybody.. BUT .. it does actually work. 

Pretty easy to scrape old room/list archives this way..if you are patient, and methodical.

Also consider the use of Add-ons like 'Remove Duplicate Messages".. that might be helpful for some of you.

Which program is this add-on for? Email client?

I use that one quite a bit.

Also be aware! MESSAGES may be LOST if the POWER GOES OUT while copying/moving!

Or due to a network outage.. so use with caution/intelligence.

 

Thu Oct 22 2020 12:27:06 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate

I'll insert my stuff in-place of your text.

Wed Oct 21 2020 17:28:50 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate

I would put the effort into your source database files first..

A). Just because you can't set c_db_shrink_files locally, doesn't mean BerkleyDB compact is out of reach.

The DB->compact() method can be reproduced with a fairly trivial c program. I am looking at some other utility packages to see if someone already has written it..

That would be great. I'd LOVE to run something to reduce database size by factor of 10000.
I happen to have citadel 8.24 source tree and was thinking of may be just adding the DB->compact call after each purge, even without using some test variable.
Can you tell me what do I need to do to build citadel?

B). "slacker method".. Drag/drop messages over IMAP. May not be terrible if both boxes are local to each other. I have done many systems this way.. admittedly nothing like 50G!

I am not quite following this "Drag/drop messages over IMAP". What exactly do I have to do? My 2 boxes are on the same lan, but it is at virtual server provider facilities.

C).  You may have the 'migrate' utility with your original version.  That's an XML exporter.  They at least you'd have a parse-able archive.

Unfortunately, I don't seem to have the 'migrate' utility, at least in /usr/local/ctdlsupport or /usr/local/citadel dirs. But I vaguely recall something about the XML export/import functionality of cit 824. Do you happen to know where I can find some info on it, if there is any? Looks like old docs, where it is described, are gone, and the new ones do not seem to mention it.

Wed Oct 21 2020 04:15:10 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, ctdlmigrate better have ssh port configurable. Not many run ssh on port 22 nowadays.

I tried to do db_recover on the database produced by Ctdlmigrate  and these are the results.

(as root)
cd /usr/local/citadel/data/
/usr/local/ctdlsupport/bin/db_recover -c

db_recover: BDB0110 Log sequence error: page LSN 1 4377; previous LSN 1 6649
db_recover: BDB1520 Recovery function for LSN 1 6761 failed on forward pass
db_recover: BDB0061 PANIC: Invalid argument
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 24
db_recover: BDB3027 cdb.02: unable to flush page: 24
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 25
db_recover: BDB3027 cdb.02: unable to flush page: 25
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.02: write failed for page 26
db_recover: BDB3027 cdb.02: unable to flush page: 26
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 2
db_recover: BDB3027 cdb.03: unable to flush page: 2
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.03: write failed for page 3
db_recover: BDB3027 cdb.03: unable to flush page: 3
db_recover: BDB0060 PANIC: fatal region error detected; run recovery
db_recover: BDB3015 cdb.0d: write failed for page 1
db_recover: BDB3027 cdb.0d: unable to flush page: 1
db_recover: BDB1544 process-private: unable to find environment
db_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I hope they tell you something. So, what's the verdict? Is there any hope for recovering this baby?

Wed Oct 21 2020 15:42:41 EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

First of all, thanx for your help. Appreciated indeed.

I'll plug in some points inline of your text.

Wed Oct 21 2020 15:17:00 EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Just continue on this point ..

/usr/local/citadel/files is file attachments, I'm fairly certain, for those rooms marked as file upload rooms.  I have occasionally seen files in mine.  Whether a file is stored internally (such as an imap attachment) or put in this directory depends on things I'm too lazy to look up right now.. but I'm inferring, direct uploads through webcit to a 'file upload room', is the main criteria.

  • I have nothing in /usr/local/citadel/files

/usr/local/citadel/messages has nothing to do with actual messages.. it's probably not even used anymore.  Mine has nothing in it. It was probably more like what is in text client messages ^nodename login/out templates. It's listed as ^bbsdir .. even so, trivially easy to copy even if permissions need to be modified..

/usr/local/citadel/keys < - keys obviously.. and keys are usually very tightly clamped down.. But also, often particular to a specific host.  In my mind, keys don't specifically need to be copied, they might be, or they might be recreated on the new system.  ctlmigrate is just trying to make it easy for you.  But if it did work, it might not actually be correct, because keys can overlap hostnames, and the new system might not have all the same domains, subdomains as the old, so that is kind of moot anyway.  I think the goal is to move all your email messages, and accounts, not to predict every situation that anyone could ever be in.

  • I copied my original keys on the top of those produced by Ctdlmigrate

If I recall correctly, there was something of a catch 22 with citadel migrate..  You needed to be root, but couldn't be root in order to do one particular thing (sorry, it's been a few years since I've used it..)

  • Yes, I can tell you exactly what needs to be done in order to make ssh work:
  • Edit /etc/ssh/sshd_config
    Change:

    Port NN to:
    Port 22

    #PermitRootLogin no
    PermitRootLogin yes

    # Change to no to disable tunnelled clear text passwords
    PasswordAuthentication yes
    #PasswordAuthentication no

    Then restart sshd
    #sudo systemctl restart sshd
    sudo service ssh restart
Therefore, I'd say, that your migration was successful.   Sometime in the future we might add a little logic around "keys" or some verbiage about permissions..

"Your migration was successful but don't forget you must recreate your keys!"

  • Yep, it does look to me that I got all the data I need. But how come the cit server does not start with those "scheduled jobs" etc.?

or just bail out early on..

"You have to change permission on blah first, please run chmod ..."

  • To change permissions on what files?

or just dumb it down. "don't forget to examine the directories /files and /messages and /keys you may need to move those by hand"..

Sounds kind of odd for a "migration utility" to  become dumber, but may be the best choice considering many unknown factors out there.

Maybe the first step should be to chmod/chown all the files on the source system temporarily..

  • Chmod/chown to what values and for what purpose?

IN ANY CASE YOUR MIGRATION PROBABLY WORKED!

  • Huraaay!!! It worked so well that I just wanna cry like a baby!!!

   rv += create_dir(ctdl_message_dir   , S_IRUSR|S_IWUSR|S_IXUSR, UID, -1);

   help_subst(buffer, "^bbsdir", ctdl_message_dir);

  •  Well, if those instructions make you happy, then there is a hope I'll get it working one of these days, I hope not in too distant of a future...
  • Thanx again
Wed Oct 21 2020 02:38:53 PM EDT from warbaby @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

Because if rsync fails, then you'll get that message.

But it's right at the very end of the process..

So, you'll have to interpret the meaning.  [It would be good to have ctdlmigrate actually echo out the full commands..but even without that.. it's obvious from "receiving incremental file list"]

check messages/

check keys/

You should have your answer right there.

ctdlmigrate.c 247-275

 

      if (!strncasecmp(buf, "files|", 6)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[6], ctdl_file_dir);
      }
      else if (!strncasecmp(buf, "messages|", 9)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[9], ctdl_message_dir);
      }
      else if (!strncasecmp(buf, "keys|", 5)) {
         snprintf(cmd, sizeof cmd, "rsync -va --rsh='ssh -S %s' %s@%s:%s/ %s/",
            socket_path, remote_user, remote_host, &buf[5], ctdl_key_dir);
      }
      else {
         strcpy(cmd, "false");   /* cheap and sleazy way to throw an error */
      }
      printf("%s\n", cmd);
      cmdexit = system(cmd);
      if (cmdexit != 0) {
         exitcode += cmdexit;
      }
   }
   pclose(sourcefp);

THEEND:  if (exitcode == 0) {
      printf("\n\n *** Citadel migration was successful! *** \n\n");
   }
   else {
      printf("\n\n *** Citadel migration was unsuccessful. *** \n\n");
   }


Wed Oct 21 2020 01:52:26 PM EDT from platonov @ Uncensored Subject: Re: Ctdlmigrate and Citadel Key files

I am using letsencrypt and I do have a script that runs "sudo certbot renew" followed by mv-certs-to-citadel.sh below.

But I am not sure I follow why ctdlmigrate does not complete OK just because of a couple of key files.

Wed Oct 21 2020 12:20:43 EDT from warbaby @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

If you want to break this out into another thread about Citadel Keys, if it is in fact the issue, it may not be a bad idea..

What I'm seeing is likely rsync failing on the two key files citadel.key and citadel.csr They are chmod 600 owned by root:staff or citadel:root

Your migration may be successful except for that.  Are you using letsencrypt?  you can just copy the files from live/ .. (can't link, wrong owner permissions)

If you're not using let's encrypt, try to recreate your self-signed keys, check the contents of everything in /usr/local/citadel/keys and (same on webcit/keys. for webcit I just use a link to citadel/keys)..

Also, if using letsencrypt, use fullchain.pem .. not sure if its still an issue but, there some browsers were throwing errors  if you didn't include the whole chain.

# mv-certs-to-citadel.sh

#!/bin/bash
if [[ $# -eq 0 ]] ; then
    echo 'I need a domain related to a cert in /etc/letsencrypt/live';
    echo "Like one of these";
    echo "`ls -alh /etc/letsencrypt/live`";
    exit 0
   fi

DOMAIN=$1;
CIT_CERT_DIR="/usr/local/citadel/keys"; # Citadel Server
WEB_CERT_DIR="/usr/local/webcit/keys"; # Webcit
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$CIT_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$CIT_CERT_DIR/citadel.key"
cp "/etc/letsencrypt/live/$DOMAIN/fullchain.pem" "$WEB_CERT_DIR/citadel.cer"
cp "/etc/letsencrypt/live/$DOMAIN/privkey.pem" "$WEB_CERT_DIR/citadel.key"
chown -R citadel:root "$CIT_CERT_DIR"
chown -R citadel:root "$WEB_CERT_DIR"

Wed Oct 21 2020 08:41:59 AM EDT from platonov @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?

Well, yesterday I tried to do /usr/local/citadel/ctdlmigrate to xfer database from 8.24 to 9.29 node.

I was able to make it work and it did xfer the data to the new node. And, Subject: headers were magically working correctly. I did verify quite a bit of records and everything looked just fine. Does it mean it is a configuration issue?

Unfortunately, ctdlmigrate did not fully succeed and it bombed out with an error msg:

*** Citadel migration was unsuccessful. ***

Here are last few lines.

- - - - - - - - - - - - - - - - -

root@45.79.97.253:/usr/local/citadel/keys/ /usr/local/citadel/keys/
receiving incremental file list

sent 20 bytes  received 85 bytes  210.00 bytes/sec
total size is 5,286  speedup is 50.34
false
false

 *** Citadel migration was unsuccessful. ***

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And the real bummer is that everything seemed to work just fine with imported data until you restart the citserver.

At that point, here's the status:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

$ sudo service citadel status
● citadel.service - Citadel Server
     Loaded: loaded (/etc/systemd/system/citadel.service; enabled; vendor preset                                                                                                                                   : enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-20 20:42:41 UTC; 2s ag                                                                                                                                   o
    Process: 1507 ExecStart=/usr/local/citadel/citserver (code=exited, status=10                                                                                                                                   5)
   Main PID: 1507 (code=exited, status=105)

Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Scheduled restart job, resta                                                                                                                                   rt counter is at 5.
Oct 20 20:42:41 UbuToo systemd[1]: Stopped Citadel Server.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Start request repeated too q                                                                                                                                   uickly.
Oct 20 20:42:41 UbuToo systemd[1]: citadel.service: Failed with result 'exit-cod                                                                                                                                   e'.
Oct 20 20:42:41 UbuToo systemd[1]: Failed to start Citadel Server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

What can be done to fix this issue? I was thinking of doing a database recover kind of thing.

Tue Oct 20 2020 23:12:29 EDT from IGnatius T Foobar @ Uncensored Subject: Re: Is there a problem of incorrect displaying of Subject: header?
What I am seeing on 9.17 is incorrect displaying or not displaying at all
of
Subject: header in RSS feed rooms.

I have noticed this on occasion as well. It's got to be a buffer size issue somewhere but so far I haven't been able to figure out where it is. We'll keep looking.

 



 



 



 



 



 



 



 



 



 



 



 



Go to page: First ... 20 21 22 23 [24] 25 26 27 28 ... Last