Citadel Support

Thu Jun 21 2012 02:31:09 EDTfrom "Robert J. Clay" <rjclay@gmail.com> Subject: Re: [Citadel Support] Re: 'Too many open files' errors after upgrade to 8.11-2 on Debian

On Wed, Jun 20, 2012 at 5:03 PM, dothebart
<dothebart@uncensored.citadel.org> wrote:

I can't do that just yet but I did do a quick look at what that's
showing now & found a large number of lines for
/var/spool/citadel/network/spoolout/trada; trada (trada.rocasa.us) is
the Citadel uplink for that particular server. I shut citadel down,
removed the file, then started it back up again. It showed back up
again after a few minutes. And, if I'm reading the logs at this end
correctly, there were several lines showing (from lsof) for that file
even after I thought it had a successful session with trada.
--
Robert J. Clay
rjclay@gmail.com

ah, thats a well known and hot trace.

current GIT Master contains several changes to the citadel networking module.

can you retest with it whether this fixes your problems?

You need to upgrade libcitadle too;

heres howto compile the debs:

http://citadel.org/doku.php?id=installation:debian#tobuildfromsource

(you need to ./bootstrap to do that with git master)

heres howto obtain a clone of the git tree:

http://citadel.org/doku.php?id=installation:sourcecode

Thu Jun 21 2012 02:31:09 EDTfrom "Robert J. Clay" <rjclay@gmail.com>

I can't do that just yet but I did do a quick look at what that's
showing now & found a large number of lines for
/var/spool/citadel/network/spoolout/trada; trada (trada.rocasa.us) is
the Citadel uplink for that particular server. ...

ah, thats a well known and hot trace.

current GIT Master contains several changes to the citadel networking
module.

can you retest with it whether this fixes your problems?

You need to upgrade libcitadle too;

Tue Jun 19 2012 08:19:07 EDT from "Jamie Jones" <yagisan@yagisan.org> Subject: Re: [Citadel Support] Re: Fails to relay email via smarthost - Citadel 8.11-2 ( Debian packages )

On Tue, 19 Jun 2012 20:22:41 +0900
"dothebart" <dothebart@uncensored.citadel.org> wrote:

we have known bugs with @ in the username/password, or when the relay
isn't listening on port  25.

can you confirm one of the both is your error cause?

Thank you for your quick response.

The relay is listening on port 25.

The relay username however is in the format of username@isp.domain

Is there a bug tracker I can search for known bugs ?

Regards,
Jamie
--
Jamie Jones
Email: yagisan@yagisan.org

GPG/PGP signed mail preferred.
PGP Key ID 0x4B6E7209
Fingerprint E1FD 9D7E 6BB4 1BD4 AEB9 3091 0027 CEFA 4B6E 7209

I've got the same problem.

The relay is listen to port 587

format for user is username@isp.domain

Debian wheezy, Citadel is 8.11-1 (not -2)

Thanks for help!

Is there a workaround, search for a ISP, which is listen to port 25?

Cheers Alex

just tested it for port 25, it  doesn't works either.

Cheers!

Alex

currently there are three possible solutions.

- downgrade to 8.05

- compile from GIT master (see howto some mails back)

- install a local postfix (or whatever) and relay through this

Hello,

How do I setup doors. I cant find any hint in the documentation.

Thanks mtg

http://www.citadel.org/doku.php?id=documentation:system_administration_manual

this one?

in general, you have to link two citadels, and then add rooms on both sides to be shared. I think the term doors isn't used there anymore.

Thu Jun 21 2012 02:31:09 EDTfrom "Robert J. Clay" <rjclay@gmail.com>

I can't do that just yet but I did do a quick look at what that's
showing now & found a large number of lines for
/var/spool/citadel/network/spoolout/trada; trada (trada.rocasa.us) is
the Citadel uplink for that particular server.

ah, thats a well known and hot trace.

current GIT Master contains several changes to the citadel networking
module.

can you retest with it whether this fixes your problems?
You need to upgrade libcitadle too;

heres howto compile the debs:

http://citadel.org/doku.php?id=installation:debian#tobuildfromsource

(you need to ./bootstrap to do that with git master)

heres howto obtain a clone of the git tree:

http://citadel.org/doku.php?id=installation:sourcecode

currently there are three possible solutions.

Jame

Fri Jun 22 2012 22:15:41 EDT from jame @ Uncensored Subject: Re: Fails to relay email via smarthost - Citadel 8.11-2 ( Debian packages )

 

- install a local postfix (or whatever) and relay through this

I already run  my system that way.  Updated the smarthosts setting to only point to localhost on the standard port.   Does anything showing in the SMTP queue need to be deleted somehow?

 Jame

they should be removed once they're sent successfull.

however, if you had problems sending mails, reattempting to send will take a while.

you can make it send them immediately on the next queue run by running

sendcommand smtp runqueue

Fri Jun 22 2012 20:45:55 EDTfrom "Robert J. Clay" <rjclay@gmail.com> Subject: Re: [Citadel Support] Re: 'Too many open files' errors after upgrade to 8.11-2 on Debian

On Thu, Jun 21, 2012 at 8:06 AM, dothebart
<dothebart@uncensored.citadel.org> wrote:

Thu Jun 21 2012 02:31:09 EDTfrom "Robert J. Clay" <rjclay@gmail.com>

I can't do that just yet but I did do a quick look at what that's
showing now & found a large number of lines for
/var/spool/citadel/network/spoolout/trada; trada (trada.rocasa.us) is
the Citadel uplink for that particular server.

ah, thats a well known and hot trace.

current GIT Master contains several changes to the citadel networking
module.

can you retest with it whether this fixes your problems?
You need to upgrade libcitadle too;

So, build libcitadel, install libcitadel-dev; then build the
versions of webcit & citadel? (Although you didn't actually say that
I need to do webcit as well...).

 

heres howto compile the debs:

http://citadel.org/doku.php?id=installation:debian#tobuildfromsource

That doesn't seem very helpful to me for building from the git
repository... Perhaps it could use another section?

Note that I am used to doing a source build from a git repo using
git-buildpackage, then using pbuilder for a binary build...

 

(you need to ./bootstrap to do that with git master)

For any builds? (debian/rules doesn't seem to call it...) Just
for a newly cloned git repo? (I don't recall if I ever ran it when I
first cloned it...)

 

heres howto obtain a clone of the git tree:

http://citadel.org/doku.php?id=installation:sourcecode

I just did a 'git pull' on the clone I originally did awhile ago;
commit bcbaf2800c778043c12f9fd2d719ca9271ac6cb4 is showing as the
most recent. Build with that? (And should I continue this in
'Citadel Development'?)

yes,  this version hopefully shouldn't loose fds anymore; plus messages should go out quicker.

Maybe Im wrong... but today all my servers shows this activity

I wonder if the hackers manage to send mails without authentication... and how

Even my server#3 have the SMTP disable for all the users except myself is just a storage mail server.

How can I fix/block/avoid this?

right now I use "/sbin/route add -net 91.201.0.0 netmask 255.255.0.0. reject"

to kick the hacker @ss

Here are the logs

server#1

 

Jun 23 08:29:30 server1 citadel: Session (SMTP-MTA) started from 91.201.64.104 (91.201.64.104).
Jun 23 08:29:30 server1 citadel: SMTP server: EHLO 127.0.0.1 <-- What ?
Jun 23 08:29:31 server1 citadel: SMTP server: AUTH LOGIN
Jun 23 08:29:31 server1 citadel: SMTP server: AUTH LOGIN
Jun 23 08:29:31 server1 citadel: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:29:31 server1 citadel: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:29:32 server1 citadel: SMTP server: mail from: testing@testers.com
Jun 23 08:29:32 server1 citadel: SMTP server: rcpt to: csclus.smtp@gmail.com
Jun 23 08:29:32 server1 citadel: SMTP server: data
Jun 23 08:29:32 server1 citadel: SMTP server: Content-Type: text/html
Jun 23 08:29:32 server1 citadel: SMTP server: From: testing@testers.com
Jun 23 08:29:32 server1 citadel: SMTP server: To: csclus.smtp@gmail.com
Jun 23 08:29:32 server1 citadel: SMTP server: Subject: virgin - xxx.xxx.xxx.xx <== IP address of my server!
Jun 23 08:29:32 server1 citadel: SMTP server: .
Jun 23 08:29:32 server1 citadel: SMTP server: QUIT
Jun 23 08:29:32 server1 citadel: [11998] Session ended.

Jun 23 08:30:55 server1 citadel: SMTP server: AUTH LOGIN
Jun 23 08:30:55 server1 citadel: POP3: RETR 11#015
Jun 23 08:30:55 server1 citadel: SMTP server: AUTH LOGIN
Jun 23 08:30:55 server1 citadel: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:30:55 server1 citadel: POP3: QUIT#015
Jun 23 08:30:55 server1 citadel: [12004] Session ended.
Jun 23 08:30:55 server1 citadel: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:30:56 server1 citadel: SMTP server: mail from: testing@testers.com
Jun 23 08:30:56 server1 citadel: SMTP server: rcpt to: csclus.smtp@gmail.com
Jun 23 08:30:56 server1 citadel: SMTP server: data
Jun 23 08:30:56 server1 citadel: SMTP server: Content-Type: text/html
Jun 23 08:30:56 server1 citadel: SMTP server: From: testing@testers.com
Jun 23 08:30:56 server1 citadel: SMTP server: To: csclus.smtp@gmail.com
Jun 23 08:30:56 server1 citadel: SMTP server: Subject: virgin - xxx.xxx.xxx.xx <== IP address of my server!
Jun 23 08:30:56 server1 citadel: SMTP server: .
Jun 23 08:30:56 server1 citadel: SMTP server: QUIT
Jun 23 08:30:56 server1 citadel: [12002] Session ended.
Jun 23 08:30:57 server1 citadel: Thread "Worker Thread" (0xffffffff9b45a700) exited.

server#2

Jun 23 08:30:38 server1 citadel: SMTP client: processing outbound queue
Jun 23 08:30:38 server1 citadel: SMTP client: queue run completed; 0 messages processed
Jun 23 08:30:38 server1 citadel: Thread "SMTP Send" (0x5f641700) exited.
Jun 23 08:30:39 server1 citadel: Garbage Collection for thread "SMTP Send" (0x7f485f641700).
Jun 23 08:30:52 server1 citadel: Session (SMTP-MTA) started from 91.201.64.104 (91.201.64.104).
Jun 23 08:30:52 server1 citadel: SMTP server: EHLO 127.0.0.1
Jun 23 08:30:52 server1 citadel: SMTP server: AUTH LOGIN
Jun 23 08:30:52 server1 citadel: SMTP server: AUTH LOGIN
Jun 23 08:30:53 server1 citadel: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:30:53 server1 citadel: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:30:53 server1 citadel: SMTP server: server1 from: testing@testers.com
Jun 23 08:30:53 server1 citadel: SMTP server: rcpt to: csclus.smtp@gmail.com
Jun 23 08:30:53 server1 citadel: SMTP server: data
Jun 23 08:30:53 server1 citadel: SMTP server: Content-Type: text/html
Jun 23 08:30:53 server1 citadel: SMTP server: From: testing@testers.com
Jun 23 08:30:53 server1 citadel: SMTP server: To: csclus.smtp@gmail.com
Jun 23 08:30:53 server1 citadel: SMTP server: Subject: virgin - xxx.xxx.xxx.xx <== IP address of my server!
Jun 23 08:30:53 server1 citadel: SMTP server: .
Jun 23 08:30:53 server1 citadel: SMTP server: QUIT
Jun 23 08:30:53 server1 citadel: [7449] Session ended.

server#2 - no smtp service enabled

Jun 23 08:26:38 noSMTPhere! citserver[14941]: No external notifiers configured on system/user
Jun 23 08:27:39 noSMTPhere! citserver[14941]: SMTPCQ: processing outbound queue
Jun 23 08:27:39 noSMTPhere! citserver[14941]: SMTPCQ: queue run completed; 0 messages processed 0 activated
Jun 23 08:27:39 noSMTPhere! citserver[14941]: No external notifiers configured on system/user
Jun 23 08:28:27 noSMTPhere! citserver[14941]: Session (SMTP-MTA) started from  (91.201.64.104).
Jun 23 08:28:27 noSMTPhere! citserver[14941]: SMTP server: EHLO 127.0.0.1
Jun 23 08:28:27 noSMTPhere! citserver[14941]: SMTP server: AUTH LOGIN
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: AUTH LOGIN
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: mail from: testing@testers.com
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: rcpt to: csclus.smtp@gmail.com
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: data
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: Content-Type: text/html
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: From: testing@testers.com
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: To: csclus.smtp@gmail.com
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: Subject: virgin - xxx.xxx.xxx.xx <== IP address of my server!
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: .
Jun 23 08:28:28 noSMTPhere! citserver[14941]: SMTP server: QUIT
Jun 23 08:28:28 noSMTPhere! citserver[14941]: Context: [160]SRV[SMTP-MTA] Session ended.
Jun 23 08:28:40 noSMTPhere! citserver[14941]: SMTPCQ: processing outbound queue
Jun 23 08:28:40 noSMTPhere! citserver[14941]: SMTPCQ: queue run completed; 0 messages processed 0 activated
Jun 23 08:28:40 noSMTPhere! citserver[14941]: No external notifiers configured on system/user
Jun 23 08:29:41 noSMTPhere! citserver[14941]: SMTPCQ: processing outbound queue
Jun 23 08:29:41 noSMTPhere! citserver[14941]: SMTPCQ: queue run completed; 0 messages processed 0 activated
Jun 23 08:29:41 noSMTPhere! citserver[14941]: No external notifiers configured on system/user
Jun 23 08:29:51 noSMTPhere! citserver[14941]: Session (SMTP-MTA) started from  (91.201.64.104).
Jun 23 08:29:51 noSMTPhere! citserver[14941]: SMTP server: EHLO 127.0.0.1
Jun 23 08:29:51 noSMTPhere! citserver[14941]: SMTP server: AUTH LOGIN
Jun 23 08:29:51 noSMTPhere! citserver[14941]: SMTP server: AUTH LOGIN
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: bnVsbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: mail from: testing@testers.com
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: rcpt to: csclus.smtp@gmail.com
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: data
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: Content-Type: text/html
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: From: testing@testers.com
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: To: csclus.smtp@gmail.com
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: Subject: virgin - xxx.xxx.xxx.xx <== IP address of my server!
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: .
Jun 23 08:29:52 noSMTPhere! citserver[14941]: SMTP server: QUIT
Jun 23 08:29:52 noSMTPhere! citserver[14941]: Context: [161]SRV[SMTP-MTA] Session ended.

Please advice

first, there is no way to make citadel an open relay; it simply doesn't implement it.

second, if you have enabled self service account creation, by default these accounts won't be able to send mail out.

maybe you should have a look at your userlist, and find some "new" user who is trying to abuse your system.

THANKS dothebart! But sometimes you're very cryptic to me.

I just wonder Why citadel accept the "DATA" and doesn't show a message
like "not allowed" or something like that. The logs looks like the mail
was accepted and mailed.

* maybe you should have a look at your userlist, and find some "new"
user who is trying to abuse your system.*- And If I did Not found
"new" users, what can I do next?**and *how in heavens, if the "self
service" is disabled, can a "new" user be added/authorized by itself?

The "*EHLO 127.0.0.1*" scares me

google for the ofender IP and the mails "from" and "to" and all of them
has been used since 2011, I found the IP logged on some other servers. I
wonder if google can check the recipient mailbox and disabled, they has
been active from many months. When I see the EHLO from some IP I just
block/reject with iptables the address but now, how to block 127.0.0.1 ?

*Question:*
I need then to change the "0.0.0.0" IP and put just the x.x.x.x public
IP in this field? but I have this kind of setup on my network:

two of my boxes have two NICs one is the public static IP and the other
NIC is the internal network.

server#1: eth0 is for 187.x.y.33 , eth1=192.168.33.50
server#2: eth0 is for 187.x.y.34 , eth1=192.168.33.51
server#3: eth0 is for 187.x.y.35 , eth1=none

you can do

echo 'this aaaa string on login'| base64 -d

to find out whose account is used; maybe s.b. was stolen the credentials?

Thanks Stefan and dothebart!

http://cbl.abuseat.org/lookup.cgi?ip=91.201.64.104

is good to see them listed, hopefully we didn't see the sc*mbag from some time. I wonder where and how to report those kind of "relay attemps". Sad, spamhaus didn't show them listed yet.

I don't replicate mails between systems, each are independent and "Allow unauthenticated SMTP clients to spoof this sites domains" is disabled on each one.

Our server1 use spamhaus, clamav and spamassassin (I add each bad server by hand)

Server 2 (used to sent invoice mails) need to send only our mails and detect rejected mails to take care of them.

In Server 3, I need to know How To drop mails that aren't from our own invoice mail system (is running inside a virtualbox) this server have the smtp service disabled to everyone, so nobody can send mails, just receive invoices, and each customer have an account to log in to webcit and download their invoices, I need to disable deleting messages, so I modify the webcit static.local files need it but I can find a way to disable the "delete" key.

I think that in our case blocking IPs can be "good" because we don't have commercial relations with some IPs from countries like Russian, China, Ucrayne that are the source of the abuse.

So my first goal is to protect our systems and the firewall can be the choice by now. Is so sad to watch some people trying to abuse the small business, really why can all we live in peace and prosper? scapes from my mind. After setting up the firewall I can't use telnet remotelly to our servers... I don't need it of course, good?!

By now I setup son firewall rules to block some IPs blocks, hope the cr*pers (I mean hackers) forget us and leave us alone.

And yes, I need to find time to learn and setup postfix as the MTA in Citadel but I wonder what can I gain doing this, I mean all works fine until the cr*ppers knock our doors.

Regards

Sat Jun 23 2012 09:30:55 PM EDT from "Stefan Schwarz" <stefan.schwarz@gmx.com> Subject: Re: *** GMX Spamverdacht *** [Citadel Support] hacked - virgin x.x.x.x - please explain a little more.

I just wonder Why citadel accept the "DATA" and doesn't show a message
like "not allowed" or something like that. The logs looks like the mail
was accepted and mailed.

If you suspect that, just try via telnet, after all SMTP is just plain
text sent between computers :-)

If you can mail to external mail-adresses without authentication you act
as an open-relay, although dothebart disagrees thats possible.

Maybe "Allow unauthenticated SMTP clients to spoof this sites domains"
is activated?

* maybe you should have a look at your userlist, and find some "new"
user who is trying to abuse your system.*- And If I did Not found
"new" users, what can I do next?**and *how in heavens, if the "self
service" is disabled, can a "new" user be added/authorized by itself?

The "*EHLO 127.0.0.1*" scares me

That does not matter much, since citadel does not check if that value is
correct or not (maybe with integrated spamassassion). The decision to
accept mails for foreign domains (open relay) is not based on that
value, but this not possible anyway.

"EHLO is just like HELO except that the server's response text provides
computer-readable information about the server's abilities."

If you want real spam checks, than put something like "policyd-weight"
in front or integrate spamassassin.

, ... in fact there is no "new" user, I

google for the ofender IP and the mails "from" and "to" and all of them
has been used since 2011, I found the IP logged on some other servers. I
wonder if google can check the recipient mailbox and disabled, they has
been active from many months. When I see the EHLO from some IP I just
block/reject with iptables the address but now, how to block 127.0.0.1 ?

A Firewall is not helping since there is only the ability to filter on
lower levels like IP-address and not on the application level.

The potential spammer is connecting from "91.201.64.104", blocking IPs
is not a good anti-spam measure.
127.0.0.1 is localhost -> not public reachable, only within one computer.

*Question:*
I need then to change the "0.0.0.0" IP and put just the x.x.x.x public
IP in this field? but I have this kind of setup on my network:

it depends, 0.0.0.0 binds all available IPs (one of them is the public IP).

two of my boxes have two NICs one is the public static IP and the other
NIC is the internal network.

server#1: eth0 is for 187.x.y.33 , eth1=192.168.33.50
server#2: eth0 is for 187.x.y.34 , eth1=192.168.33.51
server#3: eth0 is for 187.x.y.35 , eth1=none

Do you replicate mails between these hosts? I personally would use
postfix as MTA, citadel has limited abilities (by design) in this field.

greetings
Stefan