Language:
switch to room list switch to menu My folders
Go to page: First ... 23 24 25 26 [27] 28 29 30 31
[#] Mon Nov 09 2020 08:08:10 EST from platonov @ Uncensored

Subject: Re: Problem with incorrect displaying of Subject: header fixed

[Reply] [ReplyQuoted] [Headers] [Print]

 I'll comment in-line.

Sun Nov 08 2020 17:06:35 EST from IGnatius T Foobar @ Uncensored Subject: Re: Problem with incorrect displaying of Subject: header fixed
Interesting. I see that you are reporting that the problem is that the current version of the RSS reader does not RFC2047-encode the subject lines, I agree that it needs to do so, and I have confirmed that the code change belongs where you say it belongs ...

The current version of RSS reader does not handle the headers at all. Basically, that code has never been written or adopted from the earlier version. Current version of XML parser and related code is a major rewrite and the XML fields are simply stored into the SMTP headers without any processing or handling.

And the problem is not only the Subject: header. The worst part of the problem is with SMTP From: header, and proper processing/handling of that one is not as simple as one might think. There are several forms of From header. Furthermore, not all the characters or tokens are to be encoded and so on... So, I think you are probably the one who has the most expertise on that code.

but I can't get it to display improperly here.

Strange... May be you need to look at the sources of serv_rssclient.c for current version compared to 8.21. It is pretty obvious that this thing can not just magically work for you unless you have a pretty old version. I'll include both version as an attachment here. Just in case...

Btw, have you seen the fix I have sent to warbaby? I left some older code commented out just to make it easier to see the differences. It only handles the Subject: issue and From header handling has not been implemented. It would probably take you not so long to wire that code in.


What does your reading environment look like? Linux or Windows?

I am reading it via Webcit under Windows. Warbaby is also having the same problem.

Text client, web, or IMAP?

Actually, I am also using Thunderbird IMAP and everything looks correct.

Can you send a marked-up screenshot?

 Not sure what exactly do you want to see. May be warbaby would understand you better...

 



serv_rssclient_824.c (text/x-csrc, 30309 bytes) [ View | Download ]
serv_rssclient_929.c (text/x-csrc, 15729 bytes) [ View | Download ]
[#] Mon Nov 09 2020 09:05:43 EST from platonov @ Uncensored

Subject: Re: Problem with incorrect displaying of Subject: header fixed

[Reply] [ReplyQuoted] [Headers] [Print]

Well, there was a mail delivery problem with my message to warbaby that included the fix source code. So, I'll repost the attachment here to make sure you see the fix and/or try it for yourself and do a code review. I am not sure if there is some kind of problem with this post, in which case you can just remove it. I just want to make sure we're making progress on this and get the fix into the source tree.

[See serv_rssclient.c with fix in attachment]

Sun Nov 08 2020 17:06:35 EST from IGnatius T Foobar @ Uncensored Subject: Re: Problem with incorrect displaying of Subject: header fixed

Interesting. I see that you are reporting that the problem is that the current version of the RSS reader does not RFC2047-encode the subject lines, I agree that it needs to do so, and I have confirmed that the code change belongs where you say it belongs ... but I can't get it to display improperly here.

What does your reading environment look like? Linux or Windows? Text client, web, or IMAP? Can you send a marked-up screenshot?

 



serv_rssclient_fixed_01.c (text/x-csrc, 16334 bytes) [ View | Download ]
[#] Mon Nov 09 2020 09:54:19 EST from IGnatius T Foobar @ Uncensored

Subject: Re: Problem with incorrect displaying of Subject: header fixed

[Reply] [ReplyQuoted] [Headers] [Print]

Sure thing. I understand the problem but I want to make sure I'm getting the fix correct. I have code up in the editor but if yours works I'll review that and use it instead.

You're correct in observing that serv_rssclient.c is a substantial rewrite.
A few years ago we made a decision to simplify all of the pollers by letting libcurl do all the work. This allowed us to remove thousands of lines of code and two external dependencies. RFC2047-encoding of header fields is definitely something that was overlooked, so I'm glad you have brought it to our attention.

[#] Mon Nov 09 2020 12:08:14 EST from platonov @ Uncensored

Subject: Re: Problem with incorrect displaying of Subject: header fixed

[Reply] [ReplyQuoted] [Headers] [Print]

Mon Nov 09 2020 09:54:19 EST from IGnatius T Foobar @ Uncensored Subject: Re: Problem with incorrect displaying of Subject: header fixed
Sure thing. I understand the problem but I want to make sure I'm getting the fix correct. I have code up in the editor but if yours works I'll review that and use it instead.

You're correct in observing that serv_rssclient.c is a substantial rewrite.
A few years ago we made a decision to simplify all of the pollers by letting libcurl do all the work. This allowed us to remove thousands of lines of code and two external dependencies. RFC2047-encoding of header fields is definitely something that was overlooked, so I'm glad you have brought it to our attention.

Great to hear that. But I forgot to mention in my last post that this problem did not exist in textclient and the Subject headers looked good on it. After I wrote my msg I tried to run textclient again just to verify things.

And boom! Now all the rss messages that were received after I installed my fix no longer have Subject displayed correctly. But old messages, before the fix, looked good.

Not sure if textclient has been upgraded since this major rewrite related to libcurl.

As far as textclient issue goes, even before this fix you could not write a message in non-ASCII. If you switch the keyboard language and then start typing, it is all displayed in ASCII. So...

 

 



[#] Tue Nov 10 2020 13:34:30 EST from userT @ Uncensored

Subject: Webct-ng and CalDAV

[Reply] [ReplyQuoted] [Headers] [Print]

Excuse me, by chance any news about Webcit-ng and CalDAV?

There are several things broken in current Webcit. Prominently, for instance, the Calendar room.
The day view always graphically displays the hour ranges wrongly.
The "Check attendee availability" button only adds a certain amount of hours to the starting hour of the event, even if user is actually free.

Also, either I don't know how to use it, or the "Chat" room has never been functional
(testing in my local Citadel installation using latest version).



[#] Wed Nov 11 2020 12:13:10 EST from platonov @ Uncensored

Subject: Re: Problem with incorrect displaying of Subject: header fixed

[Reply] [ReplyQuoted] [Headers] [Print]

Well, this is not about the Subject header directly, but it deals with the same source file, and once we are at it, why not clean up some other things we can...

There is one not very pleasant issue with rssclient requiring a single source line change, that I'd like to ask to take care of.

The issue is: when rssclient is having a problem with some feed, it logs the message, saying:

  • "rssclient: failed to load feed: ..."

But it does not tell you WHAT is the feed's URI and which room is it in, which is a shame - to have an error that you can not take care of simply because you don't have ANY information on where and what it is.

Good logging practice is to precisely and sufficiently completely identify the key information. Simply saying: "rssclient: failed to load feed" is pretty meaningless and is a cause of unnecessary waste of time and energy.

So, I would really appreciate if we make this single source line change:

File serv_rssclient.c, Line 406:

    if (res != CURLE_OK) {
        syslog(LOG_WARNING, "rssclient: failed to load feed: %s, room %s, %s",
            url->url, url->rooms->room, curl_easy_strerror(res));
    }

Mon Nov 09 2020 09:54:19 EST from IGnatius T Foobar @ Uncensored Subject: Re: Problem with incorrect displaying of Subject: header fixed
Sure thing. I understand the problem but I want to make sure I'm getting the fix correct. I have code up in the editor but if yours works I'll review that and use it instead.

You're correct in observing that serv_rssclient.c is a substantial rewrite.
A few years ago we made a decision to simplify all of the pollers by letting libcurl do all the work. This allowed us to remove thousands of lines of code and two external dependencies. RFC2047-encoding of header fields is definitely something that was overlooked, so I'm glad you have brought it to our attention.

 



[#] Wed Nov 11 2020 21:56:18 EST from ParanoidDelusions @ Uncensored

[Reply] [ReplyQuoted] [Headers] [Print]

Is there documentation on how to set up an RSS feed as a Room on Citadel? 




[#] Wed Nov 11 2020 22:37:11 EST from Smashbot @ Uncensored

Subject: inetd/xinetd

[Reply] [ReplyQuoted] [Headers] [Print]

I read the FAQ to redirect incoming telnet connections to the citadel client,  but my Debian 10 stretch didnt come with inetd or xinet d and I cant figure out

what to do with 

telnet stream tcp4 nowait root /usr/sbin/tcpd /usr/sbin/in.telnetd -h -L /usr/local/citadel/citadel


[#] Wed Nov 11 2020 22:54:54 EST from Smashbot @ Uncensored

Subject: Re: inetd/xinetd

[Reply] [ReplyQuoted] [Headers] [Print]

 

Wed Nov 11 2020 22:37:11 EST from Smashbot @ Uncensored Subject: inetd/xinetd

I read the FAQ to redirect incoming telnet connections to the citadel client,  but my Debian 10 stretch didnt come with inetd or xinet d and I cant figure out

what to do with 

telnet stream tcp4 nowait root /usr/sbin/tcpd /usr/sbin/in.telne td -h -L /usr/local/citadel/citadel

and if I paid any attention to what I typed I would have corrected stretch and typed buster instead  :-F

 



[#] Thu Nov 12 2020 12:13:09 EST from Smashbot @ Uncensored

Subject: Re: inetd/xinetd

[Reply] [ReplyQuoted] [Headers] [Print]

I figured it out. Somehow I apt'ed both inetd and xinetd and it missed tcpd and at the end of the day I had to modify some of the syntax in inetd.conf to work. Now I just have to figure out how to do some extensive logging on the incoming connections because of the tremendous amount of scans and scripted attacks that happen on every subnet of the hosting provider I am using. :/

[#] Thu Nov 12 2020 14:54:47 EST from omatnet @ Uncensored

Subject: In Webcit 929 (latest?) Email aliases section under user contact infor

[Reply] [ReplyQuoted] [Headers] [Print]

Hi,

Using WebCit 926, Citadel 929 (Ubuntu 20.04) "Email aliases" section under user contact information is missing! I noticed that on WebCit 925 (on this support server) the email aliases are still showing as before under "Update your contact information".

I set up a fresh install of Citadel 929 on Ubuntu 20.04 to with the intention to copy manually all settings from an older server installation. But the “Internet e-mail aliases” section at the bottom of “Update your contact information” window – is completely missing. We use many email aliases an cannot operate without them.

Any suggestions?

Alternatively, is there a way to install a pervious version of WebCit, like 925? (I tried to download the source code from the Citadel website, but download doesn't complete).

Thanks!



[#] Thu Nov 12 2020 15:01:22 EST from omatnet @ Uncensored

Subject: Both methods of migration attempts crash within seconds

[Reply] [ReplyQuoted] [Headers] [Print]

Hi,

I'm trying to migrate our Citadel 8.15 on Ubuntu 12.04 LTS to Citadel 929 on Ubuntu 20.04 however both methods of migration attempts crash within seconds (FYI - I was able to migrate using both methods on other servers).

Explanation: I tried to migrate all data and settings, first using data export/import (sendcommand "ARTV export" > exported.dat) and then using ctdlmigrate (which I used successfully on other servers). In both cases the process crashes within seconds. I can add that 10 days ago I deleted over 130,000 spam messages that were received in the last years but were never actually deleted, and since then action the nightly auto-purge process crashes the citadel server every night, requiring me to restart the Citadel server every morning.

Is there a way to fix the purge process or repair any corrupted files to avoid crashing during a migration attempt? 

If not, is there any other way to migrate the Citadel server, data and settings?

 
Thank you so much for any ideas!


[#] Thu Nov 12 2020 16:07:25 EST from warbaby @ Uncensored

Subject: Re: Both methods of migration attempts crash within seconds

[Reply] [ReplyQuoted] [Headers] [Print]

We just had a migration thread like this.. and you can probably save lots of time/trouble using the slacker method.  [one guy used it successfully to migrate a 50gb mail db]

Setup a client logged into both systems.. [with perhaps Thunderbird]

give your users instructions and opportunity to drag their messages between accounts into the matching folder on their new .

rooms/llists can be migrated very easily like this.  Just make sure you have a matching room on the new system, drag the messages over imap, drop them in the new folder.

That's about it really.  Line up two webcit windows, check all your system settings before you deep six the old server.

I've done it many times.  It's a good chance to sort, prune and cleanup as you go.

Works perfectly as long as you don't lose network, or power during the process.

There are some thunderbird extensions like "remove duplicate messages" which can help also.

The final step is to cut the DNS.

If you plan ahead, you can probably do this by just updating the single "mail" (mail server) A record.

 

 

Thu Nov 12 2020 03:01:22 PM EST from omatnet @ Uncensored Subject: Both methods of migration attempts crash within seconds

Hi,

I'm trying to migrate our Citadel 8.15 on Ubuntu 12.04 LTS to Citadel 929 on Ubuntu 20.04 however both methods of migration attempts crash within seconds (FYI - I was able to migrate using both methods on other servers).

Explanation: I tried to migrate all data and settings, first using data export/import (sendcommand "ARTV export" > exported.dat) and then using ctdlmigrate (which I used successfully on other servers). In both cases the process crashes within seconds. I can add that 10 days ago I deleted over 130,000 spam messages that were received in the last years but were never actually deleted, and since then action the nightly auto-purge process crashes the citadel server every night, requiring me to restart the Citadel server every morning.

Is there a way to fix the purge process or repair any corrupted files to avoid crashing during a migration attempt? 

If not, is there any other way to migrate the Citadel server, data and settings?

 
Thank you so much for any ideas!


 



[#] Thu Nov 12 2020 16:12:23 EST from warbaby @ Uncensored

Subject: RSS Feeds in Rooms

[Reply] [ReplyQuoted] [Headers] [Print]

Create the room first. Then Advanced, Edit, and Remote Retrieval.

https://www.citadel.org/rss_aggregation.html

 

Wed Nov 11 2020 09:56:18 PM EST from ParanoidDelusions @ Uncensored

Is there documentation on how to set up an RSS feed as a Room on Citadel? 




 



[#] Thu Nov 12 2020 17:00:08 EST from ParanoidDelusions @ Uncensored

Subject: Re: RSS Feeds in Rooms

[Reply] [ReplyQuoted] [Headers] [Print]

 

Thu Nov 12 2020 16:12:23 EST from warbaby @ Uncensored Subject: RSS Feeds in Rooms

Create the room first. Then Advanced, Edit, and Remote Retrieval.

https://www.citadel.org/rss_aggregation.html

 

 

That is way easier than I expected. :) 

 



[#] Thu Nov 12 2020 17:38:03 EST from platonov @ Uncensored

Subject: Re: Both methods of migration attempts crash within seconds

[Reply] [ReplyQuoted] [Headers] [Print]

Well, migration might work if you do all the steps exactly. Try these steps:

(just make sure you do a backup of your citadel/data dir)

To export/import all Citadel databases do (this was done under Ubuntu 20.04):

 

  • sudo /usr/local/citadel/sendcommand "MIGR export" > ~/citadel/cit929_exported_2020_11_10_01.xml
  • sudo service citadel stop
  • su -
  • rm /usr/local/citadel/data/*
  • cd /usr/local/citadel
  • sudo service citadel start
  • ./setup
  • sudo /usr/local/citadel/sendcommand "MIGR import" < ~/citadel/cit929_exported_2020_11_10_01.xml
  • sudo service citadel restart

What is to keep in mind is that you don't do "migr import" to the citadel databases that still exist (meaning citadel/data/* are not removed). There are several reasons why you want to do the import to the virgin citadel databases.

Basically, with "MIGR import" you are rebuilding everything and in order to avoid any kinds of conflicts with the old data and possibly network activity, like RSS feeds or mail runs, causing database updates you do migr import on a virgin system.

This has been done several times and the last one was on Nov 10, 2020.

Hope it works for you.

Thu Nov 12 2020 16:07:25 EST from warbaby @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

We just had a migration thread like this.. and you can probably save lots of time/trouble using the slacker method.  [one guy used it successfully to migrate a 50gb mail db]

Setup a client logged into both systems.. [with perhaps Thunderbird]

give your users instructions and opportunity to drag their messages between accounts into the matching folder on their new .

rooms/llists can be migrated very easily like this.  Just make sure you have a matching room on the new system, drag the messages over imap, drop them in the new folder.

That's about it really.  Line up two webcit windows, check all your system settings before you deep six the old server.

I've done it many times.  It's a good chance to sort, prune and cleanup as you go.

Works perfectly as long as you don't lose network, or power during the process.

There are some thunderbird extensions like "remove duplicate messages" which can help also.

The final step is to cut the DNS.

If you plan ahead, you can probably do this by just updating the single "mail" (mail server) A record.

 

 

Thu Nov 12 2020 03:01:22 PM EST from omatnet @ Uncensored Subject: Both methods of migration attempts crash within seconds

Hi,

I'm trying to migrate our Citadel 8.15 on Ubuntu 12.04 LTS to Citadel 929 on Ubuntu 20.04 however both methods of migration attempts crash within seconds (FYI - I was able to migrate using both methods on other servers).

Explanation: I tried to migrate all data and settings, first using data export/import (sendcommand "ARTV export" > exported.dat) and then using ctdlmigrate (which I used successfully on other servers). In both cases the process crashes within seconds. I can add that 10 days ago I deleted over 130,000 spam messages that were received in the last years but were never actually deleted, and since then action the nightly auto-purge process crashes the citadel server every night, requiring me to restart the Citadel server every morning.

Is there a way to fix the purge process or repair any corrupted files to avoid crashing during a migration attempt? 

If not, is there any other way to migrate the Citadel server, data and settings?

 
Thank you so much for any ideas!


 



 



[#] Thu Nov 12 2020 19:45:32 EST from ParanoidDelusions @ Uncensored

Subject: Re: RSS Feeds in Rooms

[Reply] [ReplyQuoted] [Headers] [Print]

 

 
 

That is way easier than I expected. :) 

 



So, if I go crazy because my BBS is dead and add a bunch of RSS threads so I feel like I have a vibrant user base, will it fill the drive and cause a kernel panic, or is Citadel smart enough to purge from its message database if things are filling up the SD card that my Rpi is running on? 




[#] Fri Nov 13 2020 12:26:13 EST from omatnet @ Uncensored

Subject: Re: Both methods of migration attempts crash within seconds

[Reply] [ReplyQuoted] [Headers] [Print]

Thanks,

That's actually a very clever idea!
I setup on Thunderbird two IMAP accounts for the same email user, one to the old server and one to the new server (and later repeat the process for each user), and then copy all the files from the old-server account to the new-server account. Although doing it manually takes much longer than an automatic migration process and it needs to be done folder by folder, but it works very well in my case since the automatic migration process crashes the server.

Thank you so much for this solution,
cheers!

Thu Nov 12 2020 16:07:25 EST from warbaby @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

We just had a migration thread like this.. and you can probably save lots of time/trouble using the slacker method.  [one guy used it successfully to migrate a 50gb mail db]

Setup a client logged into both systems.. [with perhaps Thunderbird]

give your users instructions and opportunity to drag their messages between accounts into the matching folder on their new .

rooms/llists can be migrated very easily like this.  Just make sure you have a matching room on the new system, drag the messages over imap, drop them in the new folder.

That's about it really.  Line up two webcit windows, check all your system settings before you deep six the old server.

I've done it many times.  It's a good chance to sort, prune and cleanup as you go.

Works perfectly as long as you don't lose network, or power during the process.

There are some thunderbird extensions like "remove duplicate messages" which can help also.

The final step is to cut the DNS.

If you plan ahead, you can probably do this by just updating the single "mail" (mail server) A record.

 

 

Thu Nov 12 2020 03:01:22 PM EST from omatnet @ Uncensored Subject: Both methods of migration attempts crash within seconds

Hi,

I'm trying to migrate our Citadel 8.15 on Ubuntu 12.04 LTS to Citadel 929 on Ubuntu 20.04 however both methods of migration attempts crash within seconds (FYI - I was able to migrate using both methods on other servers).

Explanation: I tried to migrate all data and settings, first using data export/import (sendcommand "ARTV export" > exported.dat) and then using ctdlmigrate (which I used successfully on other servers). In both cases the process crashes within seconds. I can add that 10 days ago I deleted over 130,000 spam messages that were received in the last years but were never actually deleted, and since then action the nightly auto-purge process crashes the citadel server every night, requiring me to restart the Citadel server every morning.

Is there a way to fix the purge process or repair any corrupted files to avoid crashing during a migration attempt? 

If not, is there any other way to migrate the Citadel server, data and settings?

 
Thank you so much for any ideas!


 



 



[#] Fri Nov 13 2020 12:46:10 EST from omatnet @ Uncensored

Subject: Re: Both methods of migration attempts crash within seconds

[Reply] [ReplyQuoted] [Headers] [Print]

Thank you for the detailed reply!

 

I tried your suggestion, but just the same - it crashed within seconds with the last text line being "Alarm clock", and the Citadel server crashed as well. The XML file created was very small, just as if the creation stopped abruptly. Although there is no other indication, I can only assume that there is some corruption in the data file, or a bad parameter that causes the crash.

Are you familiar with any utility to check/repair the data files before any migration attemps? Does 'sendcommand' have any command that would do that? I can find a detailed list of its commands

Is there a way to debug or troubleshoot something like that? If I know what causes the problem, or even if it is a specific user - I could at least delete that user and continue with the full automatic migration.

 

Thank you so much!

Thu Nov 12 2020 17:38:03 EST from platonov @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

Well, migration might work if you do all the steps exactly. Try these steps:

(just make sure you do a backup of your citadel/data dir)

To export/import all Citadel databases do (this was done under Ubuntu 20.04):

 

  • sudo /usr/local/citadel/sendcommand "MIGR export" > ~/citadel/cit929_exported_2020_11_10_01.xml
  • sudo service citadel stop
  • su -
  • rm /usr/local/citadel/data/*
  • cd /usr/local/citadel
  • sudo service citadel start
  • ./setup
  • sudo /usr/local/citadel/sendcommand "MIGR import" < ~/citadel/cit929_exported_2020_11_10_01.xml
  • sudo service citadel restart

What is to keep in mind is that you don't do "migr import" to the citadel databases that still exist (meaning citadel/data/* are not removed). There are several reasons why you want to do the import to the virgin citadel databases.

Basically, with "MIGR import" you are rebuilding everything and in order to avoid any kinds of conflicts with the old data and possibly network activity, like RSS feeds or mail runs, causing database updates you do migr import on a virgin system.

This has been done several times and the last one was on Nov 10, 2020.

Hope it works for you.

Thu Nov 12 2020 16:07:25 EST from warbaby @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

We just had a migration thread like this.. and you can probably save lots of time/trouble using the slacker method.  [one guy used it successfully to migrate a 50gb mail db]

Setup a client logged into both systems.. [with perhaps Thunderbird]

give your users instructions and opportunity to drag their messages between accounts into the matching folder on their new .

rooms/llists can be migrated very easily like this.  Just make sure you have a matching room on the new system, drag the messages over imap, drop them in the new folder.

That's about it really.  Line up two webcit windows, check all your system settings before you deep six the old server.

I've done it many times.  It's a good chance to sort, prune and cleanup as you go.

Works perfectly as long as you don't lose network, or power during the process.

There are some thunderbird extensions like "remove duplicate messages" which can help also.

The final step is to cut the DNS.

If you plan ahead, you can probably do this by just updating the single "mail" (mail server) A record.

 

 

Thu Nov 12 2020 03:01:22 PM EST from omatnet @ Uncensored Subject: Both methods of migration attempts crash within seconds

Hi,

I'm trying to migrate our Citadel 8.15 on Ubuntu 12.04 LTS to Citadel 929 on Ubuntu 20.04 however both methods of migration attempts crash within seconds (FYI - I was able to migrate using both methods on other servers).

Explanation: I tried to migrate all data and settings, first using data export/import (sendcommand "ARTV export" > exported.dat) and then using ctdlmigrate (which I used successfully on other servers). In both cases the process crashes within seconds. I can add that 10 days ago I deleted over 130,000 spam messages that were received in the last years but were never actually deleted, and since then action the nightly auto-purge process crashes the citadel server every night, requiring me to restart the Citadel server every morning.

Is there a way to fix the purge process or repair any corrupted files to avoid crashing during a migration attempt? 

If not, is there any other way to migrate the Citadel server, data and settings?

 
Thank you so much for any ideas!


 



 



 



[#] Fri Nov 13 2020 16:23:53 EST from platonov @ Uncensored

Subject: Re: Both methods of migration attempts crash within seconds

[Reply] [ReplyQuoted] [Headers] [Print]

I'll reply in-line...

Fri Nov 13 2020 12:46:10 EST from omatnet @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

Thank you for the detailed reply!

I tried your suggestion, but just the same - it crashed within seconds with the last text line being "Alarm clock", and the Citadel server crashed as well. The XML file created was very small, just as if the creation stopped abruptly. Although there is no other indication, I can only assume that there is some corruption in the data file, or a bad parameter that causes the crash.

This does smell like a database corruption issue. Unfortunately, I am not that much competent on database corruption issues. But "MIGR export/import" should be able to recover what is good there for data. Actually, once you do "MIGR export" part, you can then look at the XML file and see how far did it get. There are several sections in the xml file, such as <config>"</config>,,,,,and

If you got at least some messages, the rest of the xml file is messages, and some could be damaged.

Basically, I'd zipped up your entire citadel/data dir regardless. If you are patient enough to follow and track things, you'll eventually find the best possible solution under the circumstances.

Are you familiar with any utility to check/repair the data files before any migration attemps?

Look at the following post by IGnatius, Da Citadel Master:
===============================================================
Database backup/restore/recover
 Mon Sep 07 2020 18:11:49 EDT from IGnatius T Foobar @ Uncensored

Subject: Re: Database corruption

It isn't common for a Citadel database to simply go bad like that, unless you ran out of disk space, or somehow got two Citadel servers running at the same time (but the software tries to prevent that).

With the server shut down, you can attempt the following repairs:

1. Use "db_recover -c" as per the Berkeley DB supplied directions, to attempt to clean the database files.

2. If that doesn't work, the "database_cleanup.sh" script will often succeed where db_recover fails. Be sure you have plenty of free disk space because this is basically a dump-and-load of your db. 
===============================================================
Does 'sendcommand' have any command that would do that? I can find a detailed list of its commands

Well, sendcommand is what everything goes through eventually, as far as I know. That is the main "entry door" to the citadel. Look at: https://citadel.org/protocol.html

You can execute any protocol command via sendcommand interface.

Is there a way to debug or troubleshoot something like that? If I know what causes the problem, or even if it is a specific user - I could at least delete that user and continue with the full automatic migration.

Well, you are in a tough situation. Too little is known at the moment. All I can suggest is do not get frustrated and try to recover as much as you can. If "MIGR export" produced at least some of your database in XML format, then you may be able to just feed that XML back to citadel via "MIGR export".

That's the advantage and beauty of a pure text data formats. They can be edited in any way you please pretty much and if there is something that is possible to recover, you can use only that part of data which looks good.

 Thank you so much!

I wish I could be more helpful than I am. But I wish you good luck and not to get too frustrated.

Thu Nov 12 2020 17:38:03 EST from platonov @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

Well, migration might work if you do all the steps exactly. Try these steps:

(just make sure you do a backup of your citadel/data dir)

To export/import all Citadel databases do (this was done under Ubuntu 20.04):

  • sudo /usr/local/citadel/sendcommand "MIGR export" > ~/citadel/cit929_exported_2020_11_10_01.xml

This is where you split this list into 2 parts, the export and import. The following lines are for the target machine.

  • sudo service citadel stop
  • su -
  • rm /usr/local/citadel/data/*
  • cd /usr/local/citadel
  • sudo service citadel start
  • ./setup
  • sudo /usr/local/citadel/sendcommand "MIGR import" < ~/citadel/cit929_exported_2020_11_10_01.xml
  • sudo service citadel restart

What is to keep in mind is that you don't do "migr import" to the citadel databases that still exist (meaning citadel/data/* are not removed). There are several reasons why you want to do the import to the virgin citadel databases.

Basically, with "MIGR import" you are rebuilding everything and in order to avoid any kinds of conflicts with the old data and possibly network activity, like RSS feeds or mail runs, causing database updates you do migr import on a virgin system.

This has been done several times and the last one was on Nov 10, 2020.

Hope it works for you.

Thu Nov 12 2020 16:07:25 EST from warbaby @ Uncensored Subject: Re: Both methods of migration attempts crash within seconds

We just had a migration thread like this.. and you can probably save lots of time/trouble using the slacker method.  [one guy used it successfully to migrate a 50gb mail db]

Setup a client logged into both systems.. [with perhaps Thunderbird]

give your users instructions and opportunity to drag their messages between accounts into the matching folder on their new .

rooms/llists can be migrated very easily like this.  Just make sure you have a matching room on the new system, drag the messages over imap, drop them in the new folder.

That's about it really.  Line up two webcit windows, check all your system settings before you deep six the old server.

I've done it many times.  It's a good chance to sort, prune and cleanup as you go.

Works perfectly as long as you don't lose network, or power during the process.

There are some thunderbird extensions like "remove duplicate messages" which can help also.

The final step is to cut the DNS.

If you plan ahead, you can probably do this by just updating the single "mail" (mail server) A record.

 

 

Thu Nov 12 2020 03:01:22 PM EST from omatnet @ Uncensored Subject: Both methods of migration attempts crash within seconds

Hi,

I'm trying to migrate our Citadel 8.15 on Ubuntu 12.04 LTS to Citadel 929 on Ubuntu 20.04 however both methods of migration attempts crash within seconds (FYI - I was able to migrate using both methods on other servers).

Explanation: I tried to migrate all data and settings, first using data export/import (sendcommand "ARTV export" > exported.dat) and then using ctdlmigrate (which I used successfully on other servers). In both cases the process crashes within seconds. I can add that 10 days ago I deleted over 130,000 spam messages that were received in the last years but were never actually deleted, and since then action the nightly auto-purge process crashes the citadel server every night, requiring me to restart the Citadel server every morning.

Is there a way to fix the purge process or repair any corrupted files to avoid crashing during a migration attempt? 

If not, is there any other way to migrate the Citadel server, data and settings?

 
Thank you so much for any ideas!


 



 



Go to page: First ... 23 24 25 26 [27] 28 29 30 31