diff options
| -rw-r--r-- | doc/TODO.detail/wal | 101 |
1 files changed, 101 insertions, 0 deletions
diff --git a/doc/TODO.detail/wal b/doc/TODO.detail/wal index 712911d2b5b..244f4f75195 100644 --- a/doc/TODO.detail/wal +++ b/doc/TODO.detail/wal @@ -2903,3 +2903,104 @@ I regret I do not currently have time to pursue further. Best Regards, Simon Riggs +From pgsql-hackers-owner+M65147=pgman=candle.pha.pa.us@postgresql.org Fri Mar 11 12:35:29 2005 +Return-path: <pgsql-hackers-owner+M65147=pgman=candle.pha.pa.us@postgresql.org> +Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71]) + by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j2BIZSk26269 + for <pgman@candle.pha.pa.us>; Fri, 11 Mar 2005 13:35:29 -0500 (EST) +Received: from localhost (unknown [200.46.204.144]) + by svr1.postgresql.org (Postfix) with ESMTP id 7C7DD545B4 + for <pgman@candle.pha.pa.us>; Fri, 11 Mar 2005 18:35:25 +0000 (GMT) +Received: from svr1.postgresql.org ([200.46.204.71]) + by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) + with ESMTP id 31351-02 for <pgman@candle.pha.pa.us>; + Fri, 11 Mar 2005 18:35:25 +0000 (GMT) +Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) + by svr1.postgresql.org (Postfix) with ESMTP id D316E545A2 + for <pgman@candle.pha.pa.us>; Fri, 11 Mar 2005 18:35:24 +0000 (GMT) +X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org +Received: from localhost (unknown [200.46.204.144]) + by svr1.postgresql.org (Postfix) with ESMTP id E70B65640C + for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; Fri, 11 Mar 2005 18:33:21 +0000 (GMT) +Received: from svr1.postgresql.org ([200.46.204.71]) + by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) + with ESMTP id 29957-07 + for <pgsql-hackers-postgresql.org@localhost.postgresql.org>; + Fri, 11 Mar 2005 18:33:19 +0000 (GMT) +Received: from hal.kabsi.at (bottom.kabsi.at [195.202.128.74]) + by svr1.postgresql.org (Postfix) with ESMTP id 9D9EE545EE + for <pgsql-hackers@postgresql.org>; Fri, 11 Mar 2005 18:33:17 +0000 (GMT) +Received: from [192.168.0.5] (h062040243020.plc.cm.kabsi.at [62.40.243.20]) + by hal.kabsi.at (8.11.1/) with ESMTP id j2BIX7a0000748810; + Fri, 11 Mar 2005 19:33:07 +0100 (CET) +Message-ID: <4231E416.4030900@cybertec.at> +Date: Fri, 11 Mar 2005 19:31:50 +0100 +From: =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <postgres@cybertec.at> +User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.3) Gecko/20040922 +X-Accept-Language: en-us, en +MIME-Version: 1.0 +To: Simon Riggs <simon@2ndquadrant.com> +cc: Tom Lane <tgl@sss.pgh.pa.us>, + Mark Cave-Ayland <m.cave-ayland@webbased.co.uk>, + pgsql-hackers@postgresql.org +Subject: Re: [HACKERS] Cost of XLogInsert CRC calculations +References: <9EB50F1A91413F4FA63019487FCD251D113169@WEBBASEDDC.webbasedltd.local> <23031.1110206390@sss.pgh.pa.us> <1110239639.6117.197.camel@localhost.localdomain> +In-Reply-To: <1110239639.6117.197.camel@localhost.localdomain> +Content-Type: text/plain; charset=us-ascii; format=flowed +Content-Transfer-Encoding: 7bit +X-Virus-Scanned: by amavisd-new at hub.org +X-Mailing-List: pgsql-hackers +Precedence: bulk +Sender: pgsql-hackers-owner@postgresql.org +X-Virus-Scanned: by amavisd-new at hub.org +Status: OR + + +> One of the things I was thinking about was whether we could use up those +> cycles more effectively. If we were to include a compression routine +> before we calculated the CRC that would +> - reduce the size of the blocks to be written, hence reduce size of xlog +> - reduce the following CRC calculation +> +> I was thinking about using a simple run-length encoding to massively +> shrink half-empty blocks with lots of zero padding, but we've already +> got code to LZW the data down also. +> +> Best Regards, Simon Riggs +> +> +> ---------------------------(end of broadcast)--------------------------- +> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org + + +Simon, + +I think having a compression routine in there could make real sense. +We have done some major I/O testing involving compression for a large +customer some time ago. We have seen that compressing / decompressing on +the fly is in MOST cases much faster than uncompressed I/O (try a simple +"cat file | ..." vs." zcat file.gz | ...") - the zcat version will be +faster on all platforms we have tried (Linux, AIX, Sun on some SAN +system, etc. ...). +Also, when building up a large database within one transaction the xlog +will eat a lot of storage - this can be quite annoying when you have to +deal with a lot of data). +Are there any technical reasons which would prevent somebody from +implementing compression? + + Best regards, + + Hans + +-- +Cybertec Geschwinde u Schoenig +Schoengrabern 134, A-2020 Hollabrunn, Austria +Tel: +43/660/816 40 77 +www.cybertec.at, www.postgresql.at + + +---------------------------(end of broadcast)--------------------------- +TIP 5: Have you checked our extensive FAQ? + + http://www.postgresql.org/docs/faq + |