On-Demand IMAP of External Accounts with IDLE/PUSH

Use this forum if you want to suggest a new feature to hMailServer. Before posting, please search the forum to confirm that it has not already been suggested.
Post Reply

Do you need this feature?

Yes
62
87%
No
9
13%
 
Total votes: 71

t0w0i7ne
New user
New user
Posts: 17
Joined: 2009-04-24 09:27

On-Demand IMAP of External Accounts with IDLE/PUSH

Post by t0w0i7ne » 2009-04-24 09:40

Any chance at getting an external account fetcher that uses IMAP and fetches from those accounts when users log in or refresh their message/folder lists?
POP3 has fallen by the wayside so POP3 external accounts are useless for accessing mail on almost any server I do not administer myself.
It would also be awesome for users to receive mail when they ask if they have any rather than possibly waiting 20-30 minutes for the fetcher to check the external accounts.
This could also provide some TDM for the bandwidth used by the fetcher so that it does not necessarily fetch from all external accounts at the same time, but rather each user at a time when they ask for their own external mail.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: On-Demand IMAP of External Accounts

Post by ^DooM^ » 2009-04-24 09:59

POP3 has fallen by the wayside
I disagree, 95% of my users still use POP3 most of the administrators I know are seeing the same trend, sure IMAP is taking off a lot more now than it once did but i would certainly not rule out POP3 any time soon.
t0w0i7ne wrote:It would also be awesome for users to receive mail when they ask
I like that idea although as you say it could be quite bandwidth heavy if a user is checking every 60 seconds and has 10 external accounts setup. Of course this should be configurable :)
t0w0i7ne wrote:does not necessarily fetch from all external accounts at the same time, but rather each user at a time when they ask for their own external mail.
And what if they have setup 20 accounts to fetch from and it takes longer than the users configured retry time? Not getting a downer on the idea i just play devils advocate when i see potential problems.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Uzi1
Normal user
Normal user
Posts: 37
Joined: 2008-11-09 01:12

Re: On-Demand IMAP of External Accounts

Post by Uzi1 » 2009-04-26 16:42

^DooM^ wrote:it could be quite bandwidth heavy if a user is checking every 60 seconds and has 10 external accounts setup.
Maybe hmailserver could use IDLE if the external server supports it? :twisted:

t0w0i7ne
New user
New user
Posts: 17
Joined: 2009-04-24 09:27

Re: On-Demand IMAP of External Accounts

Post by t0w0i7ne » 2009-04-28 23:38

^DooM^ wrote:
t0w0i7ne wrote:does not necessarily fetch from all external accounts at the same time, but rather each user at a time when they ask for their own external mail.
And what if they have setup 20 accounts to fetch from and it takes longer than the users configured retry time? Not getting a downer on the idea i just play devils advocate when i see potential problems.
Sorry for the confusion. I did not mean that each external account of an individual user would be fetched in queue but rather that the relatively chaotic manner that users check for new mail would mean that each user's mail would be fetched at relatively different times.

This does assume that you don't have everyone checking their e-mail on any regular schedule, but my understanding of the workings of hmailserver's external accounts was that it fetched mail for all users/accounts concurrently at the specified time interval.

I also have to note that I probably will not see the same issues as other admins as I plan on having somewhere in the range of 10 users maximumm, because I am running this as a personal server not for a business or organization. (That is assuming that the server is marketed to those types of uses.)

I'm a bit of a newbie with hmailserver and I am just happy I got it working with squirrelmail over a ADSL connection.

My impetus to use it came from the university webmail being rediculously slow and unreliable (200MB of space that somehow fills up 40% with a 2kB e-mail or vital attachments that float off into the ether). The university only supports IMAP access and webmail to retreive mail (no POP3). I just want to get away from a tiny quota and having to re-send e-mails 10-20 times before classmates get them (still a problem with SMTP provided by the university). So I made my own IMAP and SMTP server using this app.


IDLE along with IMAP would be AMAZING!

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: On-Demand IMAP of External Accounts

Post by ^DooM^ » 2009-04-29 01:17

I see and i too think external fetch with IMAP and IDLE would kick ass but unless it gets some serious votes it would be a while before it is implemented.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

westdam
Senior user
Senior user
Posts: 731
Joined: 2006-08-01 21:24
Location: Padova, Italy
Contact:

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by westdam » 2009-05-22 11:09

pop3 save our disk space! :D imap is very used recently but pop3 is still the preferred.

ner0
Normal user
Normal user
Posts: 72
Joined: 2009-07-06 13:04
Location: Portugal

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ner0 » 2009-11-10 18:30

External retrieval of mail using IMAP protocol is a must have!

For those who say not needing it or being fine with POP3, well either you really don't have use for it or you didn't realize it's potential.
hMailServer as been my choice for some time now. Even without external IMAP protocol still is. I'lll give you my perspective about that:
We have several people in the Sales Dep. that either work indoor as outdoor, we have several flavours for mail clients:
- Microsoft Outlook by IMAP;
- Mozilla Thunderbird by IMAP;
- RoundCube by HTTP (IMAP);

Mostly they use their own notebooks, but sometimes for some reason they cannot access their notebook either because lack of battery, forgot it at the hotel, firewalled mail ports, etc. They alternatively use the RoundCube webmail. As long as IMAP is used everywhere the same mail content is accessible everywhere by anyone, they can access sent mail, inbound mail, drafts, trash, junk and any other folder created. You cannot have this with POP3! Actually you could, but not without creating a bunch of rules that would need to be exported and implemented very often for every single new account what would end up in a headache. What about backups? Don't get me started, we backup every single e-mail that is sent /received, actually that is kind of my problem when it comes to the lack of external IMAP feature in hMailServer. For all purpose, IMAP is a must have, vote YES for IMAP! Long life to hMailServer staff, and pretty much everyone else.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by martin » 2009-11-10 18:46

External retrieval of mail using IMAP protocol is a must have! ... hMailServer as been my choice for some time now. Even without external IMAP protocol still is.
So it's not really a "Must have", but more of a "nice to have", right... :)

ner0
Normal user
Normal user
Posts: 72
Joined: 2009-07-06 13:04
Location: Portugal

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ner0 » 2009-11-10 19:18

martin wrote:
External retrieval of mail using IMAP protocol is a must have! ... hMailServer as been my choice for some time now. Even without external IMAP protocol still is.
So it's not really a "Must have", but more of a "nice to have", right... :)
Read again... Let's not get carried away, either "must have" or "nice to have" are both figures of speech.
So I'll nod my head and say "you're right, if you want to be", you seem to need affirmation.
Please stop the non-sense, go ahead and make your day by voting NO in the poll. Is it simple enough?
Edit: In case your vote counts double, then don't vote NO please :)

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ^DooM^ » 2009-11-10 21:10

You realise Martin is the Developer of hMail right and he would ultimately be adding the support for IMAP fetch to hMailServer!
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

ner0
Normal user
Normal user
Posts: 72
Joined: 2009-07-06 13:04
Location: Portugal

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ner0 » 2009-11-10 23:04

^DooM^ wrote:You realise Martin is the Developer of hMail right and he would ultimately be adding the support for IMAP fetch to hMailServer!
Obviously I didn't realise that.
Still, these topics exist because the polls exist, so the developers are prioritizing their efforts based on the community's opinion.
This poll in particular, compare the voting ratio and posting, very different. I'm just stressing the importance of IMAP in external mail retrieval from my point of view, others disagree by literally saying that POP3 is all they will ever need. Sorry if it seems flamable, I did put a smile when I said it :). Anyhow, I mentioned that I do use hMail at work, it's like an extension of my arm, crucial. With or without IMAP, hMailServer speaks for itself and, for this matter, the developer/s. I felt like saying "nice to have IMAP" would be an understatement. Moving on...

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ^DooM^ » 2009-11-11 00:39

Moving on where? I would like to see IMAP fetch as I have already said previously, would be a great feature but it's not an important one for most.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by mattg » 2009-11-11 04:54

ner0 wrote:External retrieval of mail using IMAP protocol is a must have!

For those who say not needing it or being fine with POP3, well either you really don't have use for it or you didn't realize it's potential.
hMailServer as been my choice for some time now. Even without external IMAP protocol still is. I'lll give you my perspective about that:
We have several people in the Sales Dep. that either work indoor as outdoor, we have several flavours for mail clients:
- Microsoft Outlook by IMAP;
- Mozilla Thunderbird by IMAP;
- RoundCube by HTTP (IMAP);

Mostly they use their own notebooks, but sometimes for some reason they cannot access their notebook either because lack of battery, forgot it at the hotel, firewalled mail ports, etc. They alternatively use the RoundCube webmail. As long as IMAP is used everywhere the same mail content is accessible everywhere by anyone, they can access sent mail, inbound mail, drafts, trash, junk and any other folder created. You cannot have this with POP3! Actually you could, but not without creating a bunch of rules that would need to be exported and implemented very often for every single new account what would end up in a headache. What about backups? Don't get me started, we backup every single e-mail that is sent /received, actually that is kind of my problem when it comes to the lack of external IMAP feature in hMailServer. For all purpose, IMAP is a must have, vote YES for IMAP! Long life to hMailServer staff, and pretty much everyone else.
I have many users who connect via IMAP (mostly Squirrelmail).

I have public IMAP folders, that multiple users have access to (like say the 'sales team')
I have an account setup with an EXTERNAL Account via POP3 (like say to a gmail account).

I can set rules that move a message to an IMAP public folder when downloaded. (I do this for SPAM)

All of my users have IMAP access to these messages, but I don't need hMailserver to download from external accounts via IMAP. it works perfectly well with downloading from external accounts using POP3.

I could see that downloading from an external account via IMAP could be useful, but I don't see that you need it to do what you want...

What have I missed? Why do you think that you need external accounts to download via IMAP to achieve what you want...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

westdam
Senior user
Senior user
Posts: 731
Joined: 2006-08-01 21:24
Location: Padova, Italy
Contact:

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by westdam » 2009-11-12 16:10

not to open a flame mattg, your idea is nice..unique thing this could cause to override the 4 billion limits messages processed. If I remember wrong hmailserver set a new ID when copying/moving mails into imap folder..

( cant recall the exact ammount.. maybe it's not 4 billion.. try to understand my idea hehe :D )


of course it's a limited case.. and i agree with martin.. it's a " nice to have " not a "must to have" :) ( in my opinion )

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by mattg » 2009-11-13 10:33

No flame from me!! :) It was just a suggestion to meet the requirements detailed by ner0

http://www.hmailserver.com/forum/viewto ... 217#p94217

2 Billion
Martin wrote:One thing I've noticed though is that people who start to worry about this often hasn't thought about how many messages 2 billion is. If you receive 100 000 messages a day, it will take over 50 years before you reach that limit.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

westdam
Senior user
Senior user
Posts: 731
Joined: 2006-08-01 21:24
Location: Padova, Italy
Contact:

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by westdam » 2009-11-13 11:15

no trouble! i know you wont flame and was a good suggestion too.. :)
mine was only a tought :)

moody
New user
New user
Posts: 1
Joined: 2009-12-19 19:05

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by moody » 2009-12-19 19:57

mattg wrote:I have many users who connect via IMAP (mostly Squirrelmail).

I have public IMAP folders, that multiple users have access to (like say the 'sales team')
I have an account setup with an EXTERNAL Account via POP3 (like say to a gmail account).

I can set rules that move a message to an IMAP public folder when downloaded. (I do this for SPAM)

All of my users have IMAP access to these messages, but I don't need hMailserver to download from external accounts via IMAP. it works perfectly well with downloading from external accounts using POP3.

I could see that downloading from an external account via IMAP could be useful, but I don't see that you need it to do what you want...

What have I missed? Why do you think that you need external accounts to download via IMAP to achieve what you want...
Let's imagine that the external account via POP3 you are reffering to (gmail account), has it's folders. How would you mirror this aspect through conventional POP3. How will you manage folder aspect of IMAP within POP3?

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by mattg » 2009-12-20 02:37

You are absolutely correct. If you have an existing sub-folder setup that you wish to utilise then this won't work. (This isn't what the previous poster wanted though, I was responding to their specific query)
mattg wrote:I could see that downloading from an external account via IMAP could be useful, but I don't see that you need it to do what you want...
The difference between all users connecting via IMAP directly to the gMail account in your example, over connecting to the gMail account in your example with hMailserver as an external download is what? I see that local users will have two accounts setup in their e-mail client. Other than that?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

bids
New user
New user
Posts: 1
Joined: 2010-11-10 18:29

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by bids » 2010-11-10 18:40

I would really like this, but for a different reason. We have a corporate Exchange server that is based in the US (we are in the UK). There are three problems with this server:

(1) It's running Exchange 2003, which is ancient and has rather sucky IMAP support (it doesn't work so well with our Macs)

(2) We're limited to 150 MB of space

(3) It's at the wrong end of a rather congested WAN

To solve (2) I have set up a hMailServer at our local site (very nice by the way), users then have two accounts set up - the Exchange server and the local hMailServer, using the Exchange server as an "inbox" and the hMailServer for archiving. This allows the "archives" to be backed up to tape, it also allows the archives to be synced across multiple machines (I have three!).

What I would love to do would be to have hMailServer sync over IMAP with the corporate Exchange server. This would turn it into a local caching proxy, with all the mail available in one place. The Mac users would be happy (hMailServer works really well with Mail) and the local server would be really fast.

One solution to this would be to have hMailServer pull all the mail off Exchange using POP3. The only reason not to do this is that it would prevent corporate Blackberries and Webmail from working properly. Using IMAP would allow me to triage my mail on the Exchange server using the BB, this would be synced to the hMailServer and thus to my multiple clients.

Finally, I don't work for the IT organisation in our corporation, this is all somewhat under the radar (so I'm not able to change the way the BBs work for instance).

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by mattg » 2010-11-11 02:48

Have you looked at IMAPsize or something similar - http://www.broobles.com/imapsize/

You can script IMAP account to IMAP account updates
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

alpinestars
New user
New user
Posts: 2
Joined: 2008-04-29 13:41

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by alpinestars » 2011-12-29 21:49

Hi all,

any news regarding this?
Is it planned that IMAP will be available in future versions? Is there a possibility to do a 'workaround', e.g. a process that idles and triggers a script for downloading new mails?

Thanks a lot!

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2011-12-29 21:55

alpinestars wrote:Hi all,

any news regarding this?
Is it planned that IMAP will be available in future versions? Is there a possibility to do a 'workaround', e.g. a process that idles and triggers a script for downloading new mails?

Thanks a lot!
IMAP is not POP3. You don't have persistent message ids or temporally incoming email store in IMAP. IMAP client is not something that can fetch updates without keeping same connection open. You can't keep connection open forever.

alpinestars
New user
New user
Posts: 2
Joined: 2008-04-29 13:41

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by alpinestars » 2011-12-29 22:03

dzekas wrote:
alpinestars wrote:Hi all,

any news regarding this?
Is it planned that IMAP will be available in future versions? Is there a possibility to do a 'workaround', e.g. a process that idles and triggers a script for downloading new mails?

Thanks a lot!
IMAP is not POP3. You don't have persistent message ids or temporally incoming email store in IMAP. IMAP client is not something that can fetch updates without keeping same connection open. You can't keep connection open forever.
Okay! What I want to achieve is that mails are more or less directly on my hmailserver and not stored on the mail server of my domain hoster until I am looking there via POP3 (every 10 minutes). I am not able to lower the periodic lookup and nevertheless want to use the hoster's server due to the fact that my hmailserver at home might be not available 24x7 due to DSL reconnects etc. Any idea how I can establish a kind of 'real-time' transfer? I am connected via IMAP to hmailserver via smartphone etc.

Thx!

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2011-12-29 23:00

alpinestars wrote:Any idea how I can establish a kind of 'real-time' transfer?
You can have 2 out of 3. Cheap, 24x7, something you own.

You are asking for impossible thing and overestimating 'real-time'-ness of email. If your connection does not give you 24x7 connectivity and 24x7 has higher priority than price, then don't run your email server on DSL.

DeMeTec
New user
New user
Posts: 2
Joined: 2010-07-10 13:33

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by DeMeTec » 2012-06-06 15:26

hi there,

how about a lean version of an imap4rev1 external account that only fetches the inbox via Push-IMAP ? That would totally do IMAP external account for my purposes.

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2012-06-06 20:37

DeMeTec wrote:hi there,

how about a lean version of an imap4rev1 external account that only fetches the inbox via Push-IMAP ? That would totally do IMAP external account for my purposes.
Such software won't have "keep on" server option. If you set IMAP client to fetch emails, software will have to fetch emails and delete them on server. If emails are not deleted, you will end up with duplicate emails every time uids are invalidated on server. It beats main reason for having IMAP account. POP with "keep on server" should be replaced with IMAP. IMAP with "don't keep on server" should be replaced with POP3 or email forwarder.

If you want P-IMAP support, raise separate feature request for that and point at appropriate RFCs once draft is finalized.

If you want one time synchronization with external IMAP account, imapcopy util is totally free.

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-18 01:51

Why not such a thing like a "IMAP-Router"?

It just shows the emails under the account on the hMailServer, but doesn't actually transfer the emails. If needed, this can be done by the client using rules or even at the server using server-side rules (forward to "someone", delete original mail, done). This way it's up to oneself how the the mails will be handled.

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2012-11-18 03:14

Viator wrote:Why not such a thing like a "IMAP-Router"?

It just shows the emails under the account on the hMailServer, but doesn't actually transfer the emails. If needed, this can be done by the client using rules or even at the server using server-side rules (forward to "someone", delete original mail, done). This way it's up to oneself how the the mails will be handled.
It can also be done by client with IMAP account setup for that server. Client does not need additional proxy on hmailserver to read emails on other IMAP server. You have to transfer emails to show them. Your IMAP wrapper idea looks promising, but you never though how complex implementation of such wrapper will be.

Sane people don't try to kill frogs with lasers from cosmos.

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-18 03:30

Hi!
dzekas wrote: It can also be done by client with IMAP account setup for that server. Client does not need additional proxy on hmailserver to read emails on other IMAP server. You have to transfer emails to show them. Your IMAP wrapper idea looks promising, but you never though how complex implementation of such wrapper will be.

Sane people don't try to kill frogs with lasers from cosmos.

That's what I'm doing now. But it's not realy "elegant". See this discussion why I'm running it this way, and why I would prefer a server-side IMAP solution.

http://www.hmailserver.com/forum/viewto ... =7&t=23775

Regards,
Thorsten

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2012-11-18 03:48

Viator wrote:Hi!
dzekas wrote: It can also be done by client with IMAP account setup for that server. Client does not need additional proxy on hmailserver to read emails on other IMAP server. You have to transfer emails to show them. Your IMAP wrapper idea looks promising, but you never though how complex implementation of such wrapper will be.

Sane people don't try to kill frogs with lasers from cosmos.

That's what I'm doing now. But it's not realy "elegant". See this discussion why I'm running it this way, and why I would prefer a server-side IMAP solution.
If you read this topic, you will find multiple messages telling that you can't do server side IMAP fetch. Your not elegant solution is far more elegant than any IMAP fetch wrapper will ever be.

Setup email forwarding, if you don't want to manage multiple mailboxes. That's simple, elegant and works.

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-19 18:31

dzekas wrote: Setup email forwarding, if you don't want to manage multiple mailboxes. That's simple, elegant and works.
Just want to mention, that fetchmail can do, what you say it's worthless: fetching email using IMAP idle.
Disadvantage: just under Linux, only 1 IMAP Idle account since it's not multithreaded. With some tricks, it's possible to start fetchmail several times to fetch more than 1 IMAP Idle account.

And fetchmail is a dinosaur, so any arguments why this should be a problem are invalid. If you need to know how to do it, look at fetchmail, it's open source.

So pls, be so kind and just accept our opinion and do not try to explain me I have to want to handle my email. Thanx.

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2012-11-21 20:05

Viator wrote:
dzekas wrote: Setup email forwarding, if you don't want to manage multiple mailboxes. That's simple, elegant and works.
Just want to mention, that fetchmail can do, what you say it's worthless: fetching email using IMAP idle.
I am not fetchmail person. I am IMAP/webmail person and I have already told here that IMAP client can't track emails forever. IMAP server is free to invalidate UIDs the moment connection dies. What you are saying is possible, but it misses important IMAP UIDVALIDITY gotcha

If you can prove that fetchmail can track new messages after connection is cut off, UIDs are invalidated and other IMAP client messes up Seen/Recent flags, I will learn something new about IMAP.

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-23 20:07

dzekas wrote: If you can prove that fetchmail can track new messages after connection is cut off, UIDs are invalidated and other IMAP client messes up Seen/Recent flags, I will learn something new about IMAP.
What do you think how million of other IMAP Clients handle this problem?

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by dzekas » 2012-11-23 20:49

Viator wrote:
dzekas wrote: If you can prove that fetchmail can track new messages after connection is cut off, UIDs are invalidated and other IMAP client messes up Seen/Recent flags, I will learn something new about IMAP.
What do you think how million of other IMAP Clients handle this problem?
IMAP clients don't have local copies of IMAP mailboxes. Emails and flags are stored on server. Imap clients with persistent connections don't face UID invalidation within current session as IMAP server can't invalidate UIDs on active session. IMAP clients without persistent connections use short lived connections and depend on situation where client does not make mailbox changes that require UID invalidation.

IMAP sync or local cache stores IMAP mailbox data locally. If you want to see it in action, you could try using Thunderbird with IMAP cache enabled on large shared mailbox. If conflicting IMAP connections cause UID invalidation, Thunderbird is forced to download all messages again to rebuild cache.

UID tracking problems break POP3 "keep on server" on servers without UIDL support or on mailboxes migrated from one server to another. Lack of persistent mail IDs is not a problem in IMAP. It is standard protocol feature outside of IMAP client controls.

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-24 17:24

dzekas wrote: IMAP clients don't have local copies of IMAP mailboxes.
Local copies is default from Thunderbird.

dzekas wrote: Emails and flags are stored on server. Imap clients with persistent connections don't face UID invalidation within current session as IMAP server can't invalidate UIDs on active session. IMAP clients without persistent connections use short lived connections and depend on situation where client does not make mailbox changes that require UID invalidation.

IMAP sync or local cache stores IMAP mailbox data locally. If you want to see it in action, you could try using Thunderbird with IMAP cache enabled on large shared mailbox. If conflicting IMAP connections cause UID invalidation, Thunderbird is forced to download all messages again to rebuild cache.

UID tracking problems break POP3 "keep on server" on servers without UIDL support or on mailboxes migrated from one server to another. Lack of persistent mail IDs is not a problem in IMAP. It is standard protocol feature outside of IMAP client controls.
I start to understand. It's a misunderstanding. At least I (and as I understand most others as well) do not want to keep messages in Server A AND Server B (which is hMailServer). It's about _moving_ messages from Server A to hMailServer.
It should behave like POP but faster and without polling.

The other way, having messages on both server could be handled by simply implementing a gateway/router which just routes the protocol fom Client -> hMailServer -> IMAP Server. This way the hMailServer hast not to store anything beside a foldername (in which the mails will be shown) and the login credentials of the user.

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Bill48105 » 2012-11-24 19:31

Viator,
The only easy yet 'safe' way to do imap as an external 'pull' account would be to delete & purge per check otherwise it becomes a nightmare trying to track ID's and end up writing full email client into hmail.

Since there are tools like imapcopy or about any email client to copy/move messages between servers (or something like php which has IMAP builtin), it's not likely it'd ever be built into hmail. If you have an ongoing need to have copies between servers setup CC forward copy which about all mail servers support or use pop.
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-28 13:12

There also is a ton of software doing the POP Job. Nevertheless you have build it into hMailServer.

Also running a client which is not a service is not an option. It's not an option on a workstation, since it's a workstation. It's not always turned on and if switched off other clients such as mobile phones wont get the mail.
Running a application as a logged on user on a server is under no circumstances an option.

Using forward also is no (/not always) option. I use 3 servers. A: hMailServer as main server. B: hMailServer as Backup in case A failed. C: the mailserver from my provider for the case i.e. my internetline or the modem or ... is broken and therefore A and B failed.

If using forwarding from my provider to my hMailserver to whom shall the server send the mails? The mails are on this server because A and B can not be reached.


But I understand, that you simply do not want to implement this since in your opinion there is other software to do this. But please keep in mind, that there is also other software to act as a mailserver. I like hMailServer a lot, but beside my personal taste, there is no reason not to use a Linux mailserver, just turn the mailserver on my NAS to "ON" or even use Exchange (which is free for MS Partners).

regards,
Thorsten

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ^DooM^ » 2012-11-28 19:50

Viator wrote: there is no reason not to use a Linux mailserver, just turn the mailserver on my NAS to "ON" or even use Exchange (which is free for MS Partners).
Feel free to use those then or download hMail source and add Imap fetch yourself seeing as a million other clients handle it just fine.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Viator
New user
New user
Posts: 14
Joined: 2012-11-01 19:51

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Viator » 2012-11-30 00:47

^DooM^ wrote:Feel free to use those then or download hMail source and add Imap fetch yourself seeing as a million other clients handle it just fine.
It might have been lost somewhere in here: fetchmail exists. There is no need to reinvent the wheel.

Maybe I will add threading some day if I'm really bored.

ActionHosting
New user
New user
Posts: 2
Joined: 2013-08-08 15:34

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ActionHosting » 2013-08-08 15:48

I ended up having to use ImapSync, which was a complete ballache.

Axigen (which I've just come from) had an "Automatic Migration" method, which I found really useful.

It would be great to have this a feature, but not an absolute must.

Ede_123
New user
New user
Posts: 7
Joined: 2015-04-28 03:34

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Ede_123 » 2015-05-02 03:54

I have a little bit different use case for such a feature:

My e-mail provider offers POP3 (free of charge) and IMAP (charged).
To get IMAP for free I set-up hMailServer a long time ago with my providers mail as external account pulled via POP3 (works perfectly with one downside: new mail is not immediately available but only when the external account is fetched again).

Recently my e-mail provider made very limited IMAP support available free of charge to advertise their paid services. No folders are supported, so it's virtually unusable for anything serious, but it would offer the possibility to let hMailServer pull new mails instantly!


One suggestion:
Since an exclusive IMAP pull would be difficult because of possibly changing IDs. Would it be possible to add/use and external IMAP account for IDLE and pull mails from POP3 as soon as the IMAP server notifies about new mails?

Schubbie
New user
New user
Posts: 1
Joined: 2016-01-04 05:40

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by Schubbie » 2016-01-04 05:54

Hello,
i have just installed hMailserver and want to configure the external Accounts, but more than 3 Years later as this Threat has started, i can´t find an Option like "IMAP" or "POP3 Push" for the external Account. It is important for me to receive Mails immediately. Is someone working on it?
It would be fine, if hMailserve get incomming Mails immediately and delete it from the external Account e.g. an OneWay-Sync.
Regards,
Andreas

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by martin » 2016-01-04 08:05

hMailServer can only fetch email using POP3. No plans to add anything else.
Martin Knafve

maps
New user
New user
Posts: 1
Joined: 2018-09-11 13:53

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by maps » 2018-09-11 14:37

Everyone can have their own reasons for doing this, but when the votes go to 88% some reason will have ...

An imap server that is powered from another external imap server, may not make much sense, but the truth is that it would be masterful.
There are hundreds of mail servers, but when we choose hmailserver, it is comfortable, intuitive, easy to administer, stable, etc., a function like this would be full.

I am sure that by implementing something similar to "imapsync" in your hmailserver, you will multiply your download exponentially.

palinka
Senior user
Senior user
Posts: 4455
Joined: 2017-09-12 17:57

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by palinka » 2018-09-12 10:31

maps wrote:
2018-09-11 14:37

I am sure that by implementing something similar to "imapsync" in your hmailserver, you will multiply your download exponentially.
I agree. And I bet I'm not alone in my primary reason of getting away from big tech for privacy reasons. IMAP external accounts sure would have made the transition simpler. Moving very large mailboxes is a bear.

SectorONE
New user
New user
Posts: 1
Joined: 2020-01-07 21:01

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by SectorONE » 2020-01-07 22:25

Greetings people of hMailserver community,

I am an old-school mail admin and have deep experience with software such as Sendmail, Postfix, Imap-uw, Dovecot, Squirrelmail, Roundcube, Horde running on FreeBSD systems and including Exchange on Windows Servers and Office365.

This is my important reason why Imap must be included in the “External Accounts” option.

I have 2 imap-uw servers ,3 dovecot servers and 1 Exchange server running imap and pop3 protocol.
All the servers have multiple domains and users.

I am now trying to merge away from my old systems to hMailserver and I am on the stage of syncing from old imap/pop/exchange systems to hMailserver.

I was planning to run “imaputil” from http://www.washington.edu/imap/ or “imapsync” from https://imapsync.lamiral.info/ on one of my FreeBSD hosts, but then I observed this discussion and the “External Accounts” possibility.

If hMailservers “External Accounts” had been supporting Imap I could easy merge from all other systems to my new hMailserver.

This would then be my merge-recipe:
1. Set up all domains and users on new server.
2. Send out new login information and inform all users that older account will be closed and then synced to new mail account. (Do the DNS MX change and setup Autodiscover function).
3. When users have logged in on new server, I would reset password on old accounts on old servers and use “External Accounts” to sync ALL old mail with subfolders to new empty(more or less) account on new hMailserver.

The users will then see, with their email client (Roundcube), that the old emails and folders magically appear after a short while.

The benefits on this merge solution would be:
1. I would not need to know the accounts password on the new hMailserver.
2. Me knowing the accounts password on the old server would not be obvious to the users.
3. I would not need a third host or service running the Imap-merge(imaputil/imapsync).

Conclusion: It will be invaluable to have this option in “External Accounts” as a tool for merging email from other systems.

Please include it soon since I have ambition to scale my hMailserver and takeover multiple other small and medium email servers.

Thank you all for your hard work and all that has been done with hMailserver up to this date.
Keep up the good work :)

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by mattg » 2020-01-07 23:05

martin wrote:
2016-01-04 08:05
hMailServer can only fetch email using POP3. No plans to add anything else.

This is unlikely to be added
hMailserver is open source though, and on git hub.

Please feel free to assist with the development by forking your own copy
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by jimimaseye » 2020-01-07 23:14

I don't understand why this discussion is still happening.

Ignoring the opinion of the benefits, let's just look at the mainstream technology.

Hmailserver is designed and intended to be a simple rfc compliant mail server running on windows as an alternative to other main stream mail servers; a mail server that sits behind an SMTP port (thats simple mail transfer protocol) for distributing and receiving mails. And as far as I know the main stream alternatives (dovecot exim exchange etc) doesn't have such a feature because they too expect emails too arrive by SMTP. Their primary job isn't to collect email - they receive it. So why are we trying to make a simple mail server a complicated hybrid by having it collect email via imap?

Helping with 'migration' isn't a valid reason for me. Migrations are rare and take planning. There is software out there for migration (imapcopy etc) to help just as there is specialist software or there for migrations of databases, operating systems and other server applications - all of which often require careful specialist software or procedures. There is no reason to include 'always there functionally availability for so evening that happens once every 3 years.

(This, and the others, do allow for pop3 collection but the differences and benefits compared to this imap suggestion are obvious).

In my opinion: it's not needed and should not be incorporated (it doesn't make sense) . In reality i doubt it ever will anyway probably for the reasons already discussed above.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

palinka
Senior user
Senior user
Posts: 4455
Joined: 2017-09-12 17:57

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by palinka » 2020-01-08 02:24

jimimaseye wrote:
2020-01-07 23:14
I don't understand why this discussion is still happening.
Because it's still a good and useful idea.
Ignoring the opinion of the benefits ...

... In my opinion: it's not needed and should not be incorporated (it doesn't make sense) .
Uh huh... :mrgreen:

But seriously, if hmailserver offers pop and imap, among other things, isn't it already off the mission of providing a "simple rfc compliant mail server running on windows as an alternative to other main stream mail servers; a mail server that sits behind an SMTP port (thats simple mail transfer protocol)"?

I'm pretty sure - assuming it doesn't affect the awesome stability we enjoy - that no one would object to the addition of this good and useful option.

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by jimimaseye » 2020-01-08 10:18

palinka wrote:
2020-01-08 02:24
But seriously, if hmailserver offers pop and imap, among other things, isn't it already off the mission of providing a "simple rfc compliant mail server running on windows as an alternative to other main stream mail servers; a mail server that sits behind an SMTP port (thats simple mail transfer protocol)"?
Err... no. It's a mail server. It has to offer a way of allowing clients to collect the email it has received. As do all mail servers. No point receiving email if No one can get to read it.

Servers deliver and receive (smtp). Clients view/collect to read (imap, pop).

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

palinka
Senior user
Senior user
Posts: 4455
Joined: 2017-09-12 17:57

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by palinka » 2020-01-08 13:32

jimimaseye wrote:
2020-01-08 10:18
Err... no. It's a mail server. It has to offer a way of allowing clients to collect the email it has received. As do all mail servers. No point receiving email if No one can get to read it.

Servers deliver and receive (smtp). Clients view/collect to read (imap, pop).

[Entered by mobile. Excuse my spelling.]
My point was that postfix and sendmail are truly simple mail transport agents. The simplest, in fact. Hmailserver is far beyond simple MTA already.

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by SorenR » 2020-01-08 15:22

palinka wrote:
2020-01-08 13:32
jimimaseye wrote:
2020-01-08 10:18
Err... no. It's a mail server. It has to offer a way of allowing clients to collect the email it has received. As do all mail servers. No point receiving email if No one can get to read it.

Servers deliver and receive (smtp). Clients view/collect to read (imap, pop).

[Entered by mobile. Excuse my spelling.]
My point was that postfix and sendmail are truly simple mail transport agents. The simplest, in fact. Hmailserver is far beyond simple MTA already.
Sendmail popularity has dropped from 80% in 1996 to presently 4%... Sendmail is _dead_!
Also, I saw a comparison of Postfix and Exim... Don't u ever call Postfix simple again!

Anyways... Someone in the 'nix world made FetchMail for a reason ;-)
Fetchmail is a full-featured, robust, well-documented remote-mail retrieval and forwarding utility intended to be used over on-demand TCP/IP links (such as SLIP or PPP connections). It supports every remote-mail protocol now in use on the Internet: POP2, POP3, RPOP, APOP, KPOP, all flavors of IMAP, ETRN, and ODMR. It can even support IPv6 and IPSEC.

Fetchmail retrieves mail from remote mail servers and forwards it via SMTP, so it can then be read by normal mail user agents such as mutt, elm(1) or BSD Mail. It allows all your system MTA's filtering, forwarding, and aliasing facilities to work just as they would on normal mail.
https://www.fetchmail.info/
https://sourceforge.net/projects/fetchm ... ranch_6.4/

Someone also ported it to Windows
https://nowbotflip.netlify.com/fetchmai ... nload.html

No! I have not tried it as I have no need for it. External pickup via POP3 works for me.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4455
Joined: 2017-09-12 17:57

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by palinka » 2020-01-08 16:48

SorenR wrote:
2020-01-08 15:22
Sendmail popularity has dropped from 80% in 1996 to presently 4%... Sendmail is _dead_!
Also, I saw a comparison of Postfix and Exim... Don't u ever call Postfix simple again!
Bu..bu...but... postfix don't do no poppin' n imapin'... that would be outside the "simple" scope. Hmailserver is already "more complicated" because it be doin da poppin n imapin.
No! I have not tried it as I have no need for it. External pickup via POP3 works for me.
Me too, but hmailserver built in imap fetch would have been nice for the few account migrations I did.

I suppose, by its very nature, it could be used for a lot more than simple fetching. Sync 2 accounts between servers at all times. Could be useful.

Conclusion:

Necessary? No.
Useful? Yes.
Simple? No.

:mrgreen:

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by SorenR » 2020-01-08 19:34

Hmm... FetchMail for Windows was a hoax, it appears.

GetMail is a similar beast and it's written in Python so I presume it can be run on a Windows platform.

http://pyropus.ca/software/getmail/docu ... quirements

Oh and by the way... Found this (probably not very GDPR compliant) site today...

https://imapsync.lamiral.info/X/
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4455
Joined: 2017-09-12 17:57

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by palinka » 2020-01-08 20:14

SorenR wrote:
2020-01-08 19:34

Oh and by the way... Found this (probably not very GDPR compliant) site today...

https://imapsync.lamiral.info/X/
Wow. Somebody appears to trust them.

Image

ner0
Normal user
Normal user
Posts: 72
Joined: 2009-07-06 13:04
Location: Portugal

Re: On-Demand IMAP of External Accounts with IDLE/PUSH

Post by ner0 » 2020-01-08 20:20

palinka wrote:
2020-01-08 20:14
SorenR wrote:
2020-01-08 19:34

Oh and by the way... Found this (probably not very GDPR compliant) site today...

https://imapsync.lamiral.info/X/
Wow. Somebody appears to trust them.
It could be the webmaster using it himself... otherwise I don't understand why someone would use the online page instead of the actual tool, especially for batch migrations (and speed, possibly).

Post Reply