path: root/doc/TODO.detail/thread

From mscott@sacadia.com Wed Nov 15 14:50:19 2000
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id OAA11583
	for <pgman@candle.pha.pa.us>; Wed, 15 Nov 2000 14:50:13 -0500 (EST)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id LAA09998;
	Wed, 15 Nov 2000 11:35:33 -0800 (PST)
Date: Wed, 15 Nov 2000 11:35:33 -0800 (PST)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>,
        Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
Subject: Please help with some advice
Message-ID: <Pine.GSO.4.10.10011151053260.9940-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Status: ORr

Dear Sirs,

I have been lurking on the PostgreSQL hackers list for about 3 months now
and your names comes up more than any with helpful info about the project
so I was hoping you could help me.

Let me cut to the chase.  I have been experimenting with 7.0.2 source to
see if I could create a mutlti-threaded version of the backend so
I could link directly from java ( I have a fe<->be protocol that I use for
my apps). Needless to say I got into much more than I bargained for.  I
now have a version that works and it has some nice benefits that are very
helpful to a project that I am working on.  What I gained was

prepared statements outside of spi
batched commits (fsync)
one connection per thread
	multiple threads per process
		multiple processes per installation

I never really intended for anyone else to see the work so I drifted
pretty far from the original code.  I also ended up using Solaris threads
rather than pthreads,  I did my own implementation of the bufmgr.c and
gram.y, and used Solaris implementation of mutex in place of S_LOCK and
TAS. I grabbed all global variables and put them in an environment
variable that is thread local.  I also did some really stupid
things like making TransactionId uint64 and making all my inserts use the
same oid.

My question is this.  I would like to get some critical feedback and
suggestions about the work from others.  What is the best way to go about
this?  I thought about trying to create a project on greatbridge.org
but I am rather new to open source and the code needs commented properly
and cleaned up before too many try and look at it.

Any suggestions would be greatly appreciated.


Thanks in advance,

Myron Scott



From mscott@sacadia.com Thu Nov 16 17:19:45 2000
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id RAA04315
	for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 17:19:43 -0500 (EST)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id OAA11449;
	Thu, 16 Nov 2000 14:05:15 -0800 (PST)
Date: Thu, 16 Nov 2000 14:05:15 -0800 (PST)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>, Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: Please help with some advice
In-Reply-To: <200011160533.AAA27886@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.10.10011161401570.11441-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Status: OR

Bruce Momjian wrote:

>I am curious how you isolated each thread.   It seems we pretty much
>assume all our memory is controlled by a single query in the process.
                                                                              

I moved all global variables to a thread global variable which is accessed
by the method GetEnv().  Which looks like this

Env* GetEnv(void) {
	Env* env;
	thr_getspecific(*envkey,(void*)&env);
	return env;
}

The Env struct includes the CurrentMemoryContext, TopMemoryContext,
PortalHeapMemory for each instance of a connection (one thread per
connection). So, for example,
EndPortalAllocMode uses GetEnv()->CurrentMemoryContext

void
EndPortalAllocMode()
{
	PortalHeapMemory context;

	AssertState(PortalManagerEnabled);
	AssertState(IsA(GetEnv()->CurrentMemoryContext,
PortalHeapMemory));

	context = (PortalHeapMemory) GetEnv()->CurrentMemoryContext;
	AssertState(PointerIsValid(context->block));		/* XXX
Trap(...) */

	/* free current mode */
	AllocSetReset(&HEAPMEMBLOCK(context)->setData);
	MemoryContextFree((MemoryContext)
PortalHeapMemoryGetVariableMemory(context),
					  context->block);

	/* restore previous mode */
	context->block = FixedStackPop(&context->stackData);
}




From vmikheev@SECTORBASE.COM Thu Nov 16 17:23:22 2000
Received: from sectorbase2.sectorbase.com ([208.48.122.131])
	by candle.pha.pa.us (8.9.0/8.9.0) with SMTP id RAA04562
	for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 17:23:21 -0500 (EST)
Received: by sectorbase2.sectorbase.com with Internet Mail Service (5.5.2650.21)
	id <V8XQB5RW>; Thu, 16 Nov 2000 14:05:24 -0800
Message-ID: <8F4C99C66D04D4118F580090272A7A234D318D@sectorbase1.sectorbase.com>
From: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>
To: "'Myron Scott'" <mscott@sacadia.com>,
        Bruce Momjian
  <pgman@candle.pha.pa.us>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Subject: RE: Please help with some advice
Date: Thu, 16 Nov 2000 14:09:30 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain;
	charset="iso-8859-1"
Status: ORr

I think the question do we want to make backend multy-threaded
should be discussed in hackers.

Vadim

> -----Original Message-----
> From: Myron Scott [mailto:mscott@sacadia.com]
> Sent: Thursday, November 16, 2000 2:05 PM
> To: Bruce Momjian
> Cc: Mikheev, Vadim; Tom Lane
> Subject: Re: Please help with some advice
> 
> 
> Bruce Momjian wrote:
> 
> >I am curious how you isolated each thread.   It seems we pretty much
> >assume all our memory is controlled by a single query in the process.
>                                                               
>                 
> 
> I moved all global variables to a thread global variable 
> which is accessed
> by the method GetEnv().  Which looks like this
> 
> Env* GetEnv(void) {
> 	Env* env;
> 	thr_getspecific(*envkey,(void*)&env);
> 	return env;
> }
> 
> The Env struct includes the CurrentMemoryContext, TopMemoryContext,
> PortalHeapMemory for each instance of a connection (one thread per
> connection). So, for example,
> EndPortalAllocMode uses GetEnv()->CurrentMemoryContext
> 
> void
> EndPortalAllocMode()
> {
> 	PortalHeapMemory context;
> 
> 	AssertState(PortalManagerEnabled);
> 	AssertState(IsA(GetEnv()->CurrentMemoryContext,
> PortalHeapMemory));
> 
> 	context = (PortalHeapMemory) GetEnv()->CurrentMemoryContext;
> 	AssertState(PointerIsValid(context->block));		/* XXX
> Trap(...) */
> 
> 	/* free current mode */
> 	AllocSetReset(&HEAPMEMBLOCK(context)->setData);
> 	MemoryContextFree((MemoryContext)
> PortalHeapMemoryGetVariableMemory(context),
> 					  context->block);
> 
> 	/* restore previous mode */
> 	context->block = FixedStackPop(&context->stackData);
> }
> 
> 
> 

From mscott@sacadia.com Thu Nov 16 22:16:38 2000
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id WAA14638
	for <pgman@candle.pha.pa.us>; Thu, 16 Nov 2000 22:16:36 -0500 (EST)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id TAA11874;
	Thu, 16 Nov 2000 19:04:48 -0800 (PST)
Date: Thu, 16 Nov 2000 19:04:48 -0800 (PST)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: "Mikheev, Vadim" <vmikheev@SECTORBASE.COM>, Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: Please help with some advice
In-Reply-To: <200011170156.UAA11438@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.10.10011161904140.11870-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Status: ORr

Thanks very much, I will post to hackers.

Myron



From pgsql-hackers-owner+M2691@postgresql.org Tue Jan  2 00:30:20 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA08195
	for <pgman@candle.pha.pa.us>; Tue, 2 Jan 2001 00:30:19 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f025UjL33335;
	Tue, 2 Jan 2001 00:30:45 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M2691@postgresql.org)
Received: from mailsys01.intnet.net (tmail.wwc.com [198.252.32.143] (may be forged))
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f025UTL33232
	for <pgsql-hackers@postgresql.org>; Tue, 2 Jan 2001 00:30:32 -0500 (EST)
	(envelope-from mscott@sacadia.com)
Received: from [206.112.108.0] (HELO sacadia.com)
  by mailsys01.intnet.net (CommuniGate Pro SMTP 3.3.2)
  with ESMTP id 2214231; Tue, 02 Jan 2001 00:29:47 -0500
Message-ID: <3A5167DB.3050807@sacadia.com>
Date: Mon, 01 Jan 2001 21:32:11 -0800
From: Myron Scott <mscott@sacadia.com>
Reply-To: mscott@sacadia.com
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; m18) Gecko/20001108 Netscape6/6.0
X-Accept-Language: en
MIME-Version: 1.0
To: "Ross J. Reedstrom" <reedstrm@rice.edu>
CC: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Using Threads?
References: <004401c058fd$fd498d40$f2356880@tracy> <Pine.GSO.4.10.10012032351040.28161-100000@goldengate.kojoworldwide.com.> <20001204113307.B5871@rice.edu>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

For anyone interested,

I have posted my multi-threaded version of PostgreSQL here.

http://www.sacadia.com/mtpg.html

It is based on 7.0.2 and the TAO CORBA ORB which is here.

http://www.cs.wustl.edu/~schmidt/TAO.html

Myron Scott
mkscott@sacadia.com



From bright@fw.wintelcom.net Tue Jan  2 03:02:28 2001
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA16169
	for <pgman@candle.pha.pa.us>; Tue, 2 Jan 2001 03:02:27 -0500 (EST)
Received: (from bright@localhost)
	by fw.wintelcom.net (8.10.0/8.10.0) id f0282Vm10623;
	Tue, 2 Jan 2001 00:02:31 -0800 (PST)
Date: Tue, 2 Jan 2001 00:02:31 -0800
From: Alfred Perlstein <bright@wintelcom.net>
To: Bruce Momjian <pgman@candle.pha.pa.us>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Assuming that TAS() will succeed the first time is verboten
Message-ID: <20010102000230.C19572@fw.wintelcom.net>
References: <9850.978067943@sss.pgh.pa.us> <200101020759.CAA15836@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <200101020759.CAA15836@candle.pha.pa.us>; from pgman@candle.pha.pa.us on Tue, Jan 02, 2001 at 02:59:20AM -0500
Status: OR

* Bruce Momjian <pgman@candle.pha.pa.us> [010101 23:59] wrote:
> > Alfred Perlstein <bright@wintelcom.net> writes:
> > > One trick that may help is calling sched_yield(2) on a lock miss,
> > > it's a POSIX call and quite new so you'd need a 'configure' test
> > > for it.
> > 
> > The author of the current s_lock code seems to have thought that
> > select() with a zero delay would do the equivalent of sched_yield().
> > I'm not sure if that's true on very many kernels, if indeed any...
> > 
> > I doubt we could buy much by depending on sched_yield(); if you want
> > to assume POSIX facilities, ISTM you might as well go for user-space
> > semaphores and forget the whole TAS mechanism.
> 
> 
> Another issue is that sched_yield brings in the pthreads library/hooks
> on some OS's, which we certainly want to avoid.

I know it's a major undertaking, but since the work is sort of done,
have you guys considered the port to solaris threads and seeing about
making a pthreads port of that?

I know it would probably get you considerable gains under Windows
at the expense of dropping some really really legacy system.

Or you could do what apache (is rumored) does and have it do either
threads or processes or both...

-- 
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

From pgsql-hackers-owner+M4275@postgresql.org Mon Feb  5 21:45:00 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA09262
	for <pgman@candle.pha.pa.us>; Mon, 5 Feb 2001 21:44:59 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f162ixx00920;
	Mon, 5 Feb 2001 21:44:59 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M4275@postgresql.org)
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f162fSx00595
	for <pgsql-hackers@postgresql.org>; Mon, 5 Feb 2001 21:41:29 -0500 (EST)
	(envelope-from mscott@sacadia.com)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id SAA03298
	for <pgsql-hackers@postgresql.org>; Mon, 5 Feb 2001 18:25:05 -0800 (PST)
Date: Mon, 5 Feb 2001 18:25:05 -0800 (PST)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Using Threads?
Message-ID: <Pine.GSO.4.10.10102051823210.3289-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

I have put a new version of my multi-threaded
postgresql experiment at

http://www.sacadia.com/mtpg.html

This one actually works.  I have added a server
based on omniORB, a CORBA 2.3 ORB from ATT.  It
   is much smaller than TAO and uses the thread per
connection model.  I haven't added the java side
of the JNI interface yet but the C++ side is there.

It's still not stable but it is much better than
the last.

Myron Scott
mkscott@sacadia.com





From pgsql-hackers-owner+M4304@postgresql.org Tue Feb  6 10:24:21 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA22027
	for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 10:24:20 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16FOBx97182;
	Tue, 6 Feb 2001 10:24:11 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M4304@postgresql.org)
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16FLWx96814
	for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 10:21:33 -0500 (EST)
	(envelope-from mscott@sacadia.com)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id HAA04170;
	Tue, 6 Feb 2001 07:05:04 -0800 (PST)
Date: Tue, 6 Feb 2001 07:05:04 -0800 (PST)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: Karel Zak <zakkr@zf.jcu.cz>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Using Threads
In-Reply-To: <Pine.LNX.3.96.1010206101030.20355B-100000@ara.zf.jcu.cz>
Message-ID: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> 
>  Sorry I haven't time to see and test your experiment,
> but I have a question. How you solve memory management?
> The current mmgr is based on global variable 
> CurrentMemoryContext that is very often changed and used.
>  Use you for this locks? If yes it is probably problematic
> point for perfomance.
> 
> 			Karel
> 

There are many many globals I had to work around including all the memory
management stuff.  I basically threw everything into and "environment"
variable which I stored in a thread specific using thr_setspecific.

Performance is acually very good for what I am doing.  I was able to batch
commit transactions which cuts down on fsync calls, use prepared
statements from my client using CORBA, and the various locking calls for
the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast.  I did
some performance tests for inserts 

20 clients, 900 inserts per client, 1 insert per transaction, 4 different
tables.

7.0.2    About    10:52 average completion
multi-threaded    2:42 average completion
7.1beta3          1:13 average completion

If I increased the number of inserts per transaction, multi-threaded got
closer to 7.1 for inserts.  I haven't tested other other types of
commands
yet.


Myron Scott
mkscott@sacadia.com


From pgsql-hackers-owner+M4313@postgresql.org Tue Feb  6 12:32:00 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id MAA29163
	for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 12:31:59 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16HVox17454;
	Tue, 6 Feb 2001 12:31:51 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M4313@postgresql.org)
Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16HV6x17323
	for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 12:31:06 -0500 (EST)
	(envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
	by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id SAA03980;
	Tue, 6 Feb 2001 18:31:02 +0100
Date: Tue, 6 Feb 2001 18:31:02 +0100 (CET)
From: Karel Zak <zakkr@zf.jcu.cz>
To: Myron Scott <mscott@sacadia.com>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Using Threads
In-Reply-To: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
Message-ID: <Pine.LNX.3.96.1010206182112.3799B-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


On Tue, 6 Feb 2001, Myron Scott wrote:

> There are many many globals I had to work around including all the memory
> management stuff.  I basically threw everything into and "environment"
> variable which I stored in a thread specific using thr_setspecific.

 Yes, it's good. I working on multi-thread application server
(http://mape.jcu.cz) and I use for this project some things from PG (like
mmgr), I planning use same solution.

> Performance is acually very good for what I am doing.  I was able to batch
> commit transactions which cuts down on fsync calls, use prepared
> statements from my client using CORBA, and the various locking calls for
> the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast.  I did
> some performance tests for inserts 
> 
> 20 clients, 900 inserts per client, 1 insert per transaction, 4 different
> tables.
> 
> 7.0.2    About    10:52 average completion
> multi-threaded    2:42 average completion
> 7.1beta3          1:13 average completion

It is very very good for time for 7.1, already look forward to 7.2! :-)  

 BTW, I not sure if you anytime in future will see threads in 
official PostgreSQL and if you spending time on relevant things (IMHO).

		Karel






From pgsql-hackers-owner+M4304@postgresql.org Tue Feb  6 10:24:21 2001
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA22027
	for <pgman@candle.pha.pa.us>; Tue, 6 Feb 2001 10:24:20 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id f16FOBx97182;
	Tue, 6 Feb 2001 10:24:11 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M4304@postgresql.org)
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id f16FLWx96814
	for <pgsql-hackers@postgresql.org>; Tue, 6 Feb 2001 10:21:33 -0500 (EST)
	(envelope-from mscott@sacadia.com)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id HAA04170;
	Tue, 6 Feb 2001 07:05:04 -0800 (PST)
Date: Tue, 6 Feb 2001 07:05:04 -0800 (PST)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: Karel Zak <zakkr@zf.jcu.cz>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Using Threads
In-Reply-To: <Pine.LNX.3.96.1010206101030.20355B-100000@ara.zf.jcu.cz>
Message-ID: <Pine.GSO.4.10.10102060650250.4153-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


> 
>  Sorry I haven't time to see and test your experiment,
> but I have a question. How you solve memory management?
> The current mmgr is based on global variable 
> CurrentMemoryContext that is very often changed and used.
>  Use you for this locks? If yes it is probably problematic
> point for perfomance.
> 
> 			Karel
> 

There are many many globals I had to work around including all the memory
management stuff.  I basically threw everything into and "environment"
variable which I stored in a thread specific using thr_setspecific.

Performance is acually very good for what I am doing.  I was able to batch
commit transactions which cuts down on fsync calls, use prepared
statements from my client using CORBA, and the various locking calls for
the threads (cond_wait,mutex_lock, and sema_wait) seem pretty fast.  I did
some performance tests for inserts 

20 clients, 900 inserts per client, 1 insert per transaction, 4 different
tables.

7.0.2    About    10:52 average completion
multi-threaded    2:42 average completion
7.1beta3          1:13 average completion

If I increased the number of inserts per transaction, multi-threaded got
closer to 7.1 for inserts.  I haven't tested other other types of
commands
yet.


Myron Scott
mkscott@sacadia.com


From lamar.owen@wgcr.org Thu Jun 28 11:14:10 2001
Return-path: <lamar.owen@wgcr.org>
Received: from www.wgcr.org (IDENT:root@www.wgcr.org [206.74.232.194])
	by candle.pha.pa.us (8.10.1/8.10.1) with ESMTP id f5SFE9U18758
	for <pgman@candle.pha.pa.us>; Thu, 28 Jun 2001 11:14:09 -0400 (EDT)
Received: from lowen.wgcr.org (IDENT:lowen@[10.1.2.3])
	by www.wgcr.org (8.9.3/8.9.3/WGCR) with SMTP id LAA11879;
	Thu, 28 Jun 2001 11:14:14 -0400
Content-Type: text/plain;
  charset="iso-8859-1"
From: Lamar Owen <lamar.owen@wgcr.org>
To: Bruce Momjian <pgman@candle.pha.pa.us>
Subject: Process weight (was:Re: [GENERAL] Re: Red Hat to support PostgreSQL)
Date: Thu, 28 Jun 2001 11:14:09 -0400
X-Mailer: KMail [version 1.2]
References: <200106272258.f5RMwIb26959@candle.pha.pa.us>
In-Reply-To: <200106272258.f5RMwIb26959@candle.pha.pa.us>
MIME-Version: 1.0
Message-ID: <01062811140902.01118@lowen.wgcr.org>
Content-Transfer-Encoding: 8bit
Status: ORr

On Wednesday 27 June 2001 18:58, Bruce Momjian wrote:
> > I had almost given up on using Postgres for this system because under
> > Solaris, it just couldn't cut it (MySQL could do the work with one CPU
> > while Postgres took up even more CPU and required *both* CPUs to be
> > enabled), but when we moved the system to a Linux box, things worked
> > much better.

> Ah, back to a PostgreSQL topic.  :-)

> My guess on this one is that Solaris is slower for PostgreSQL because
> process switching is _much_ heavier on Solaris than other OS's.  This is
> because of the way they implemented processes in SVr4.  They got quite
> heavy, almost requiring kernel threads so you weren't switching
> processes all the time.

Now, the question of the week:
Is supporting a thread model for an inefficient OS a desirable thing to do, 
when more efficient OS kernels are available such as FreeBSD 4.x and Linux 
2.4?  My opinion is that our existing model, when used with a 
connection-pooling frontend, is rather efficient.  (Yes, I use a 
connection-pooling frontend.  Performance is rather nice, and I don't have to 
have a full backend spawned for every page hit.)

In fact, on a Linux box threads show as processes.  While I know that the 
kernel actually supports themin a slightly different manner than processes, 
they have more similarities than differences.

However, even on OS's where threads are supported, the mechanism to support 
those threads must be an efficient one -- not all pthreads libraries are 
created equal.  Many are frontends (expensive ones, at that) for plain old 
processes.

Does anyone know of a resource that details the 'weight' of processes for our 
supported platforms?  [reply off-list -- I'll be glad to summarize responses 
to HACKERS, ADMIN, or PORTS, as appropriate, if desired.]
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

From pgsql-hackers-owner+M13599=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 17:25:32 2001
Return-path: <pgsql-hackers-owner+M13599=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8QLPWo07589
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 17:25:32 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8QLPf405606
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 16:25:41 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13599=candle.pha.pa.us=pgman@postgresql.org)
Received: from gromit.dotclick.com (ipn9-f8366.net-resource.net [216.204.83.66])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8QKj3h82020
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 16:45:03 -0400 (EDT)
	(envelope-from markw@mohawksoft.com)
Received: from mohawksoft.com (IDENT:markw@localhost.localdomain [127.0.0.1])
	by gromit.dotclick.com (8.9.3/8.9.3) with ESMTP id QAA23693;
	Wed, 26 Sep 2001 16:43:02 -0400
Message-ID: <3BB23DD6.E86AF327@mohawksoft.com>
Date: Wed, 26 Sep 2001 16:43:02 -0400
From: mlw <markw@mohawksoft.com>
X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.4.2 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: "D. Hageman" <dhageman@dracken.com>,
   "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal
References: <Pine.LNX.4.33.0109261330030.1906-100000@typhon.dracken.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

"D. Hageman" wrote:

> The plan for the new spinlocks does look like it has some potential.  My
> only comment in regards to permformance when we start looking at SMP
> machines is ... it is my belief that getting a true threaded backend may
> be the only way to get the full potential out of SMP machines.  I see that
> is one of the things to experiment with on the TODO list and I have seen
> some people have messed around already with this using Solaris threads.
> It should probably be attempted with pthreads if PostgreSQL is going to
> keep some resemblance of cross-platform compatibility.  At that time, it
> would probably be easier to go in and clean up some stuff for the
> implementation of other TODO items (put in the base framework for more
> complex future items) as threading the backend would take a little bit of
> ideology shift.

I can only think of two objectives for threading. (1) running the various
connections in their own thread instead of their own process. (2) running
complex queries across multiple threads.

For  item (1) I see no value to this. It is a lot of work with no tangible
benefit. If you have an old fashion pthreads implementation, it will hurt
performance because are scheduled within the single process's time slice.. If
you have a newer kernel scheduled implementation, then you will have the same
scheduling as separate processes. The only thing you will need to do is
switch your brain from figuring out how to share data, to trying to figure
out how to isolate data. A multithreaded implementation lacks many of the
benefits and robustness of a multiprocess implementation.

For item (2) I can see how that could speed up queries in a low utilization
system, and that would be cool, but in a server that is under load, threading
the queries probably be less efficient.


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M13604=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 18:40:26 2001
Return-path: <pgsql-hackers-owner+M13604=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8QMePo13437
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 18:40:25 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8QMeZ417944
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 17:40:35 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13604=candle.pha.pa.us=pgman@postgresql.org)
Received: from foghorn.airs.com (foghorn.airs.com [63.201.54.26])
	by postgresql.org (8.11.3/8.11.4) with SMTP id f8QM59h01247
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 18:05:09 -0400 (EDT)
	(envelope-from ian@airs.com)
Received: (qmail 10089 invoked by uid 10); 26 Sep 2001 22:04:49 -0000
Received: (qmail 6837 invoked by uid 269); 26 Sep 2001 22:04:41 -0000
Mail-Followup-To: markw@mohawksoft.com,
  pgsql-hackers@postgresql.org,
  dhageman@dracken.com
To: "D. Hageman" <dhageman@dracken.com>
cc: mlw <markw@mohawksoft.com>,
   "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal
References: <Pine.LNX.4.33.0109261600100.1784-100000@typhon.dracken.com>
From: Ian Lance Taylor <ian@airs.com>
Date: 26 Sep 2001 15:04:41 -0700
In-Reply-To: <Pine.LNX.4.33.0109261600100.1784-100000@typhon.dracken.com>
Message-ID: <si8zf1vcau.fsf@daffy.airs.com>
Lines: 45
User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

"D. Hageman" <dhageman@dracken.com> writes:

> > you have a newer kernel scheduled implementation, then you will have the same
> > scheduling as separate processes. The only thing you will need to do is
> > switch your brain from figuring out how to share data, to trying to figure
> > out how to isolate data. A multithreaded implementation lacks many of the
> > benefits and robustness of a multiprocess implementation.
> 
> Save for the fact that the kernel can switch between threads faster then 
> it can switch processes considering threads share the same address space, 
> stack, code, etc.  If need be sharing the data between threads is much 
> easier then sharing between processes. 

When using a kernel threading model, it's not obvious to me that the
kernel will switch between threads much faster than it will switch
between processes.  As far as I can see, the only potential savings is
not reloading the pointers to the page tables.  That is not nothing,
but it is also not a lot.

> I can't comment on the "isolate data" line.  I am still trying to figure 
> that one out.

Sometimes you need data which is specific to a particular thread.
Basically, you have to look at every global variable in the Postgres
backend, and determine whether to share it among all threads or to
make it thread-specific.  In other words, you have to take extra steps
to isolate the data within the thread.  This is the reverse of the
current situation, in which you have to take extra steps to share data
among all backend processes.

> That last line is a troll if I every saw it ;-)  I will agree that threads 
> isn't for everything and that it has costs just like everything else.  Let 
> me stress that last part - like everything else.  Certain costs exist in 
> the present model, nothing is - how should we say ... perfect.

When writing in C, threading inevitably loses robustness.  Erratic
behaviour by one thread, perhaps in a user defined function, can
subtly corrupt the entire system, rather than just that thread.  Part
of defensive programming is building barriers between different parts
of a system.  Process boundaries are a powerful barrier.

(Actually, though, Postgres is already vulnerable to erratic behaviour
because any backend process can corrupt the shared buffer pool.)

Ian

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M13605=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 18:54:58 2001
Return-path: <pgsql-hackers-owner+M13605=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8QMsvo14061
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 18:54:57 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8QMt7420740
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 17:55:07 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13605=candle.pha.pa.us=pgman@postgresql.org)
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8QMOPh04333
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 18:24:26 -0400 (EDT)
	(envelope-from mscott@sacadia.com)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id PAA00633
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 15:03:00 -0700 (PDT)
Date: Wed, 26 Sep 2001 15:03:00 -0700 (PDT)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Spinlock performance improvement proposal
In-Reply-To: <3BB23DD6.E86AF327@mohawksoft.com>
Message-ID: <Pine.GSO.4.10.10109261428340.563-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR



On Wed, 26 Sep 2001, mlw wrote:

> I can only think of two objectives for threading. (1) running the various
> connections in their own thread instead of their own process. (2) running
> complex queries across multiple threads.
> 

I did a multi-threaded version of 7.0.2 using Solaris threads about a year
ago in order to try
and get multiple backend connections working under one java process using
jni.  I used the thread per connection model.

I eventually got it working, but it was/is very messy ( there were global
variables everywhere! ).  Anyway, I was able to get a pretty good speed up
on inserts by scheduling buffer writes from multiple connections on one
common writing thread.  

I also got some other features that were important to me at the time.

1.  True prepared statements under java with bound input and output
variables
2.  Better system utilization 
	a.  fewer Solaris lightweight processes mapped to threads.
	b.  Fewer open files per postgres installation 
3.  Automatic vacuums when system activity is low by a daemon thread.

but there were some drawbacks...  One rogue thread or bad user 
function could take down all connections for that process.  This
was and seems to still be the major drawback to using threads.


Myron Scott
mscott@sacadia.com


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M13602=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 17:45:26 2001
Return-path: <pgsql-hackers-owner+M13602=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8QLjQo08483
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 17:45:26 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8QLjY409914
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 16:45:35 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13602=candle.pha.pa.us=pgman@postgresql.org)
Received: from typhon.dracken.com (dv07m61.lawrence.ks.us [24.124.61.35])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8QLGDh91021
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 17:16:13 -0400 (EDT)
	(envelope-from dhageman@dracken.com)
Received: from localhost (dhageman@localhost)
	by typhon.dracken.com (8.11.4/8.11.4) with ESMTP id f8QLEMY01973;
	Wed, 26 Sep 2001 16:14:22 -0500
X-Authentication-Warning: typhon.dracken.com: dhageman owned process doing -bs
Date: Wed, 26 Sep 2001 16:14:22 -0500 (CDT)
From: "D. Hageman" <dhageman@dracken.com>
To: mlw <markw@mohawksoft.com>
cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal
In-Reply-To: <3BB23DD6.E86AF327@mohawksoft.com>
Message-ID: <Pine.LNX.4.33.0109261600100.1784-100000@typhon.dracken.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: ORr

On Wed, 26 Sep 2001, mlw wrote:
> 
> I can only think of two objectives for threading. (1) running the various
> connections in their own thread instead of their own process. (2) running
> complex queries across multiple threads.
> 
> For  item (1) I see no value to this. It is a lot of work with no tangible
> benefit. If you have an old fashion pthreads implementation, it will hurt
> performance because are scheduled within the single process's time slice..

Old fashion ... as in a userland library that implements POSIX threads?  
Well, I would agree.  However, most *modern* implementations are done in 
the kernel or kernel and userland coop model and don't have this 
limitation (as you mention later in your e-mail).  You have kinda hit on 
one of my gripes about computers in general.  At what point in time does 
one say something is obsolete or too old to support anymore - that it 
hinders progress instead of adding a "feature"?

> you have a newer kernel scheduled implementation, then you will have the same
> scheduling as separate processes. The only thing you will need to do is
> switch your brain from figuring out how to share data, to trying to figure
> out how to isolate data. A multithreaded implementation lacks many of the
> benefits and robustness of a multiprocess implementation.

Save for the fact that the kernel can switch between threads faster then 
it can switch processes considering threads share the same address space, 
stack, code, etc.  If need be sharing the data between threads is much 
easier then sharing between processes. 

I can't comment on the "isolate data" line.  I am still trying to figure 
that one out.

That last line is a troll if I every saw it ;-)  I will agree that threads 
isn't for everything and that it has costs just like everything else.  Let 
me stress that last part - like everything else.  Certain costs exist in 
the present model, nothing is - how should we say ... perfect.

> For item (2) I can see how that could speed up queries in a low utilization
> system, and that would be cool, but in a server that is under load, threading
> the queries probably be less efficient.

Well, I don't follow your logic and you didn't give any substance to back 
up your claim.  I am willing to listen.

Another thought ... Oracle uses threads doesn't it or at least it has a 
single processor and multi-processor version last time I knew ... which do 
they claim is better?  (Not saying that Oracle's proclimation of what is 
good and what is not matters, but it is good for another view point).

-- 
//========================================================\\
||  D. Hageman                    <dhageman@dracken.com>  ||
\\========================================================//


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

From pgsql-hackers-owner+M13607=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 19:14:59 2001
Return-path: <pgsql-hackers-owner+M13607=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8QNExo15536
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 19:14:59 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8QNF8423944
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 18:15:09 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13607=candle.pha.pa.us=pgman@postgresql.org)
Received: from belphigor.mcnaught.org ([216.151.155.121])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8QMe3h07256
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 18:40:04 -0400 (EDT)
	(envelope-from doug@wireboard.com)
Received: (from doug@localhost)
	by belphigor.mcnaught.org (8.11.6/8.9.3) id f8QMdkB05502;
	Wed, 26 Sep 2001 18:39:46 -0400
X-Authentication-Warning: belphigor.mcnaught.org: doug set sender to doug@wireboard.com using -f
To: "D. Hageman" <dhageman@dracken.com>
cc: mlw <markw@mohawksoft.com>,
   "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal
References: <Pine.LNX.4.33.0109261600100.1784-100000@typhon.dracken.com>
From: Doug McNaught <doug@wireboard.com>
Date: 26 Sep 2001 18:39:44 -0400
In-Reply-To: "D. Hageman"'s message of "Wed, 26 Sep 2001 16:14:22 -0500 (CDT)"
Message-ID: <m3y9n11sr3.fsf@belphigor.mcnaught.org>
Lines: 26
User-Agent: Gnus/5.0806 (Gnus v5.8.6) XEmacs/21.1 (20 Minutes to Nikko)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

"D. Hageman" <dhageman@dracken.com> writes:

> Save for the fact that the kernel can switch between threads faster then 
> it can switch processes considering threads share the same address space, 
> stack, code, etc.  If need be sharing the data between threads is much 
> easier then sharing between processes. 

This depends on your system.  Solaris has a huge difference between
thread and process context switch times, whereas Linux has very little 
difference (and in fact a Linux process context switch is about as
fast as a Solaris thread switch on the same hardware--Solaris is just
a pig when it comes to process context switching). 

> I can't comment on the "isolate data" line.  I am still trying to figure 
> that one out.

I think his point is one of clarity and maintainability.  When a
task's data is explicitly shared (via shared memory of some sort) it's
fairly clear when you're accessing shared data and need to worry about
locking.  Whereas when all data is shared by default (as with threads)
it's very easy to miss places where threads can step on each other.

-Doug
-- 
In a world of steel-eyed death, and men who are fighting to be warm,
Come in, she said, I'll give you shelter from the storm.    -Dylan

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M13611=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 21:05:02 2001
Return-path: <pgsql-hackers-owner+M13611=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8R152o22010
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 21:05:02 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8R158430261
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 20:05:08 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13611=candle.pha.pa.us=pgman@postgresql.org)
Received: from sss.pgh.pa.us ([192.204.191.242])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8R0lgh29430
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 20:47:42 -0400 (EDT)
	(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id f8R0kpK14707;
	Wed, 26 Sep 2001 20:46:51 -0400 (EDT)
To: Ian Lance Taylor <ian@airs.com>
cc: "D. Hageman" <dhageman@dracken.com>, mlw <markw@mohawksoft.com>,
   "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal 
In-Reply-To: <si8zf1vcau.fsf@daffy.airs.com> 
References: <Pine.LNX.4.33.0109261600100.1784-100000@typhon.dracken.com> <si8zf1vcau.fsf@daffy.airs.com>
Comments: In-reply-to Ian Lance Taylor <ian@airs.com>
	message dated "26 Sep 2001 15:04:41 -0700"
Date: Wed, 26 Sep 2001 20:46:51 -0400
Message-ID: <14704.1001551611@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

Ian Lance Taylor <ian@airs.com> writes:
> (Actually, though, Postgres is already vulnerable to erratic behaviour
> because any backend process can corrupt the shared buffer pool.)

Not to mention the other parts of shared memory.

Nonetheless, our experience has been that cross-backend failures due to
memory clobbers in shared memory are very infrequent --- certainly far
less often than we see localized-to-a-backend crashes.  Probably this is
because the shared memory is (a) small compared to the rest of the
address space and (b) only accessed by certain specific modules within
Postgres.

I'm convinced that switching to a thread model would result in a
significant degradation in our ability to recover from coredump-type
failures, even given the (implausible) assumption that we introduce no
new bugs during the conversion.  I'm also *un*convinced that such a
conversion will yield significant performance benefits, unless we
introduce additional cross-thread dependencies (and more fragility
and lock contention) by tactics such as sharing catalog caches across
threads.

			regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

From pgsql-hackers-owner+M13616=candle.pha.pa.us=pgman@postgresql.org Wed Sep 26 23:10:52 2001
Return-path: <pgsql-hackers-owner+M13616=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8R3Aqo03180
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 23:10:52 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8R3B3438816
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 22:11:03 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13616=candle.pha.pa.us=pgman@postgresql.org)
Received: from spider.pilosoft.com (p55-222.acedsl.com [160.79.55.222])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8R2vCh48923
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 22:57:12 -0400 (EDT)
	(envelope-from alex@pilosoft.com)
Received: from localhost (alexmail@localhost)
	by spider.pilosoft.com (8.9.3/8.9.3) with ESMTP id WAA27630;
	Wed, 26 Sep 2001 22:58:41 -0400 (EDT)
Date: Wed, 26 Sep 2001 22:58:41 -0400 (EDT)
From: Alex Pilosov <alex@pilosoft.com>
To: "D. Hageman" <dhageman@dracken.com>
cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal
In-Reply-To: <Pine.LNX.4.33.0109261733050.2225-100000@typhon.dracken.com>
Message-ID: <Pine.BSO.4.10.10109262249480.14740-100000@spider.pilosoft.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

On Wed, 26 Sep 2001, D. Hageman wrote:

> > > Save for the fact that the kernel can switch between threads faster then 
> > > it can switch processes considering threads share the same address space, 
> > > stack, code, etc.  If need be sharing the data between threads is much 
> > > easier then sharing between processes. 
> > 
> > When using a kernel threading model, it's not obvious to me that the
> > kernel will switch between threads much faster than it will switch
> > between processes.  As far as I can see, the only potential savings is
> > not reloading the pointers to the page tables.  That is not nothing,
> > but it is also
<major snippage>
> > > I can't comment on the "isolate data" line.  I am still trying to figure 
> > > that one out.
> > 
> > Sometimes you need data which is specific to a particular thread.
> 
> When you need data that is specific to a thread you use a TSD (Thread 
> Specific Data).  
Which Linux does not support with a vengeance, to my knowledge.

As a matter of fact, quote from Linus on the matter was something like
"Solution to slow process switching is fast process switching, not another
kernel abstraction [referring to threads and TSD]". TSDs make
implementation of thread switching complex, and fork() complex.

The question about threads boils down to: Is there far more data that is
shared than unshared? If yes, threads are better, if not, you'll be
abusing TSD and slowing things down. 

I believe right now, postgresql' model of sharing only things that need to
be shared is pretty damn good. The only slight problem is overhead of
forking another backend, but its still _fast_.

IMHO, threads would not bring large improvement to postgresql.

 Actually, if I remember, there was someone who ported postgresql (I think
it was 6.5) to be multithreaded with major pain, because the requirement
was to integrate with CORBA. I believe that person posted some benchmarks
which were essentially identical to non-threaded postgres...

-alex


---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

From pgsql-hackers-owner+M13619=candle.pha.pa.us=pgman@postgresql.org Thu Sep 27 00:32:55 2001
Return-path: <pgsql-hackers-owner+M13619=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8R4Wto07075
	for <pgman@candle.pha.pa.us>; Thu, 27 Sep 2001 00:32:55 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8R4X7444942
	for <pgman@candle.pha.pa.us>; Wed, 26 Sep 2001 23:33:07 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13619=candle.pha.pa.us=pgman@postgresql.org)
Received: from sss.pgh.pa.us ([192.204.191.242])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8R4Jsh61257
	for <pgsql-hackers@postgresql.org>; Thu, 27 Sep 2001 00:19:54 -0400 (EDT)
	(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.11.4/8.11.4) with ESMTP id f8R4JLK15406;
	Thu, 27 Sep 2001 00:19:21 -0400 (EDT)
To: "D. Hageman" <dhageman@dracken.com>
cc: Alex Pilosov <alex@pilosoft.com>,
   "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal 
In-Reply-To: <Pine.LNX.4.33.0109262224040.1173-100000@typhon.dracken.com> 
References: <Pine.LNX.4.33.0109262224040.1173-100000@typhon.dracken.com>
Comments: In-reply-to "D. Hageman" <dhageman@dracken.com>
	message dated "Wed, 26 Sep 2001 22:41:39 -0500"
Date: Thu, 27 Sep 2001 00:19:20 -0400
Message-ID: <15403.1001564360@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

"D. Hageman" <dhageman@dracken.com> writes:
> If you look at Myron Scott's post today you will see that it had other 
> advantages going for it (like auto-vacuum!) and disadvantages ... rogue 
> thread corruption (already debated today).

But note that Myron did a number of things that are (IMHO) orthogonal
to process-to-thread conversion, such as adding prepared statements,
a separate thread/process/whateveryoucallit for buffer writing, ditto
for vacuuming, etc.  I think his results cannot be taken as indicative
of the benefits of threads per se --- these other things could be
implemented in a pure process model too, and we have no data with which
to estimate which change bought how much.

Threading certainly should reduce the context switch time, but this
comes at the price of increased overhead within each context (since
access to thread-local variables is not free).  It's by no means
obvious that there's a net win there.

			regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

From pgsql-hackers-owner+M13621=candle.pha.pa.us=pgman@postgresql.org Thu Sep 27 01:59:44 2001
Return-path: <pgsql-hackers-owner+M13621=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8R5xio11898
	for <pgman@candle.pha.pa.us>; Thu, 27 Sep 2001 01:59:44 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8R5xi449748
	for <pgman@candle.pha.pa.us>; Thu, 27 Sep 2001 00:59:45 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13621=candle.pha.pa.us=pgman@postgresql.org)
Received: from goldengate.kojoworldwide.com. ([216.133.4.130])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8R5joh75612
	for <pgsql-hackers@postgresql.org>; Thu, 27 Sep 2001 01:45:50 -0400 (EDT)
	(envelope-from mscott@sacadia.com)
Received: from localhost (localhost [127.0.0.1])
	by goldengate.kojoworldwide.com. (8.9.1b+Sun/8.9.2) with ESMTP id WAA01144
	for <pgsql-hackers@postgresql.org>; Wed, 26 Sep 2001 22:24:29 -0700 (PDT)
Date: Wed, 26 Sep 2001 22:24:29 -0700 (PDT)
From: Myron Scott <mscott@sacadia.com>
X-Sender: mscott@goldengate.kojoworldwide.com.
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Spinlock performance improvement proposal 
In-Reply-To: <15403.1001564360@sss.pgh.pa.us>
Message-ID: <Pine.GSO.4.10.10109262146500.1111-100000@goldengate.kojoworldwide.com.>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


 
> But note that Myron did a number of things that are (IMHO) orthogonal

yes, I did :)

> to process-to-thread conversion, such as adding prepared statements,
> a separate thread/process/whateveryoucallit for buffer writing, ditto
> for vacuuming, etc.  I think his results cannot be taken as indicative
> of the benefits of threads per se --- these other things could be
> implemented in a pure process model too, and we have no data with which
> to estimate which change bought how much.
> 

If you are comparing just process vs. thread, I really don't think I
gained much for performance and ended up with some pretty unmanageable
code.

The one thing that led to most of the gains was scheduling all the writes
to one thread which, as noted by Tom,  you could do on the process model.
Besides, Most of the advantage in doing this was taken away with the
addition of WAL in 7.1.

The other real gain that I saw with threading was limiting the number of
open files but
that led me to alter much of the file manager in order to synchronize
access to the files which probably slowed things a bit.

To be honest, I don't think I, personally,
would try this again. I went pretty far off
the beaten path with this thing.  It works well for what I am doing 
( a limited number of SQL statements run many times over ) but there
probably was a better way.  I'm thinking now that I should have tried to 
add a CORBA interface for connections. I would have been able to 
accomplish my original goals without creating a deadend for myself.


Thanks all for a great project,

Myron
mscott@sacadia.com


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

From pgsql-hackers-owner+M13632=candle.pha.pa.us=pgman@postgresql.org Thu Sep 27 10:21:22 2001
Return-path: <pgsql-hackers-owner+M13632=candle.pha.pa.us=pgman@postgresql.org>
Received: from server1.pgsql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id f8RELLo08607
	for <pgman@candle.pha.pa.us>; Thu, 27 Sep 2001 10:21:21 -0400 (EDT)
Received: from postgresql.org (webmail.postgresql.org [216.126.85.28])
	by server1.pgsql.org (8.11.6/8.11.6) with ESMTP id f8RELP487000
	for <pgman@candle.pha.pa.us>; Thu, 27 Sep 2001 09:21:26 -0500 (CDT)
	(envelope-from pgsql-hackers-owner+M13632=candle.pha.pa.us=pgman@postgresql.org)
Received: from gromit.dotclick.com (ipn9-f8366.net-resource.net [216.204.83.66])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id f8RE49h21870
	for <pgsql-hackers@postgresql.org>; Thu, 27 Sep 2001 10:04:09 -0400 (EDT)
	(envelope-from markw@mohawksoft.com)
Received: from mohawksoft.com (IDENT:markw@localhost.localdomain [127.0.0.1])
	by gromit.dotclick.com (8.9.3/8.9.3) with ESMTP id KAA24417;
	Thu, 27 Sep 2001 10:02:06 -0400
Message-ID: <3BB3315D.EC99FF65@mohawksoft.com>
Date: Thu, 27 Sep 2001 10:02:05 -0400
From: mlw <markw@mohawksoft.com>
X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.4.2 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: "D. Hageman" <dhageman@dracken.com>
cc: Ian Lance Taylor <ian@airs.com>,
   "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Spinlock performance improvement proposal
References: <Pine.LNX.4.33.0109261733050.2225-100000@typhon.dracken.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR

"D. Hageman" wrote:

> On 26 Sep 2001, Ian Lance Taylor wrote:
> >
> > > Save for the fact that the kernel can switch between threads faster then
> > > it can switch processes considering threads share the same address space,
> > > stack, code, etc.  If need be sharing the data between threads is much
> > > easier then sharing between processes.
> >
> > When using a kernel threading model, it's not obvious to me that the
> > kernel will switch between threads much faster than it will switch
> > between processes.  As far as I can see, the only potential savings is
> > not reloading the pointers to the page tables.  That is not nothing,
> > but it is also not a lot.
>
> It is my understanding that avoiding a full context switch of the
> processor can be of a significant advantage.  This is especially important
> on processor architectures that can be kinda slow at doing it (x86). I
> will admit that most modern kernels have features that assist software
> packages utilizing the forking model (copy on write for instance).  It is
> also my impression that these do a good job.  I am the kind of guy that
> looks towards the future (as in a year, year and half or so) and say that
> processors will hopefully get faster at context switching and more and
> more kernels will implement these algorithms to speed up the forking
> model.  At the same time, I see more and more processors being shoved into
> a single box and it appears that the threads model works better on these
> type of systems.

"context" switching happens all the time on a multitasking system. On the x86
processor, a context switch happens when you call into the kernel. You have to go
through a call-gate to get to a lower privilege ring. "context" switching is very
fast. The operating system dictates how heavy or light a process switch is. Under
Linux (and I believe FreeBSD with Linux threads, or version 4.x ) threads and
processes are virtually identical. The only difference is that the virtual memory
pages are not "copy on write." Process vs thread scheduling is also virtually
identical.

If you look to the future, then you should accept that process switching should
become more efficient as the operating systems improve.

>
> > > I can't comment on the "isolate data" line.  I am still trying to figure
> > > that one out.
> >
> > Sometimes you need data which is specific to a particular thread.
>
> When you need data that is specific to a thread you use a TSD (Thread
> Specific Data).

Yes, but Postgres has many global variables. The assumption has always been that
it is a stand-alone process with an explicitly shared paradigm, not implicitly.

>
> > Basically, you have to look at every global variable in the Postgres
> > backend, and determine whether to share it among all threads or to
> > make it thread-specific.
>
> Yes, if one was to implement threads into PostgreSQL I would think that
> some re-writing would be in order of several areas.  Like I said before,
> give a person a chance to restructure things so future TODO items wouldn't
> be so hard to implement.  Personally, I like to stay away from global
> variables as much as possible.  They just get you into trouble.

In real live software, software which lives from year to year with active
development, things do get messy. There are always global variables involved in a
program. Efforts, of course, should be made to keep them to a minimum, but the
reality is that they always happen.

Also, the very structure of function calls may need to change when going from a
process model to a threaded model. Functions never before reentrant are now be
reentrant, think about that. That is a huge undertaking. Every single function
may need to be examined for thread safety, with little benefit.

>
> > > That last line is a troll if I every saw it ;-)  I will agree that threads
> > > isn't for everything and that it has costs just like everything else.  Let
> > > me stress that last part - like everything else.  Certain costs exist in
> > > the present model, nothing is - how should we say ... perfect.
> >
> > When writing in C, threading inevitably loses robustness.  Erratic
> > behaviour by one thread, perhaps in a user defined function, can
> > subtly corrupt the entire system, rather than just that thread.  Part
> > of defensive programming is building barriers between different parts
> > of a system.  Process boundaries are a powerful barrier.
>
> I agree with everything you wrote above except for the first line.  My
> only comment is that process boundaries are only *truely* a powerful
> barrier if the processes are different pieces of code and are not
> dependent on each other in crippling ways.  Forking the same code with the
> bug in it - and only 1 in 5 die - is still 4 copies of buggy code running
> on your system ;-)

This is simply not true. All software has bugs, it is an undeniable fact. Some
bugs are more likely to be hit than others. 5 processes , when one process hits a
bug, that does not mean the other 4 will hit the same bug. Obscure bugs kill
software all the time, the trick is to minimize the impact. Software is not
perfect, assuming it can be is a mistake.





---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

From pgsql-hackers-owner+M16320=candle.pha.pa.us=pgman@postgresql.org Thu Dec  6 10:16:20 2001
Return-path: <pgsql-hackers-owner+M16320=candle.pha.pa.us=pgman@postgresql.org>
Received: from west.navpoint.com (west.navpoint.com [207.106.42.13])
	by candle.pha.pa.us (8.11.6/8.10.1) with ESMTP id fB6FGJZ29347
	for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 10:16:19 -0500 (EST)
Received: from rs.postgresql.org (server1.pgsql.org [64.39.15.238] (may be forged))
	by west.navpoint.com (8.11.6/8.10.1) with ESMTP id fB6FGIE25797
	for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 10:16:18 -0500 (EST)
Received: from postgresql.org (postgresql.org [64.49.215.8])
	by rs.postgresql.org (8.11.6/8.11.6) with ESMTP id fB6F8MR55154
	for <pgman@candle.pha.pa.us>; Thu, 6 Dec 2001 09:12:12 -0600 (CST)
	(envelope-from pgsql-hackers-owner+M16320=candle.pha.pa.us=pgman@postgresql.org)
Received: from dendrite.sacadia.com (adsl-64-168-22-137.dsl.snfc21.pacbell.net [64.168.22.137])
	by postgresql.org (8.11.3/8.11.4) with ESMTP id fB3NNfm32380
	for <pgsql-hackers@postgresql.org>; Mon, 3 Dec 2001 18:23:42 -0500 (EST)
	(envelope-from mkscott@sacadia.com)
Received: from sacadia.com (localhost [127.0.0.1])
	by dendrite.sacadia.com (8.10.2+Sun/8.10.2) with ESMTP id fB3NKiK16816
	for <pgsql-hackers@postgresql.org>; Mon, 3 Dec 2001 15:20:44 -0800 (PST)
Message-ID: <3C0C08CC.917CC04B@sacadia.com>
Date: Mon, 03 Dec 2001 15:20:44 -0800
From: mkscott@sacadia.com
X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.8 sun4u)
X-Accept-Language: en
MIME-Version: 1.0
To: PostgreSQL Hackers Mailing List <pgsql-hackers@postgresql.org>
Subject: [HACKERS] Using Threads (again)
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


Hi All,


Since I last posted to this list I have done some work
on a multi-threaded port of Postgres 7.0.2 that I have been kicking
around for a while.  There has been some mild interest
in this in the past so I thought I might try and start a sourceforge
project with what I have so far.

>From past discussions, it is clear to me that a direct port
of postgres which uses threads instead of processes is not a
good idea, how about an embedded version that uses threads.
A multi-threaded postgres might be good for that.
The version I am working on is slower in terms of transaction
throughput than the current postgres but it uses less system
resources and does not require shared memory.

I know it is possible to embed the current postgres but I
believe that is a single user system.

Comments?


Myron Scott
mkscott@sacadia.com


---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly