Document tuple ordering differences as a possible cause of

regression test 'failures'.

2 files changed, 60 insertions, 2 deletions

diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 068b2c54eab..a4fdc5eed91 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -1,4 +1,4 @@
-<!-- $Header: /cvsroot/pgsql/doc/src/sgml/regress.sgml,v 1.15 2000/12/22 18:57:50 petere Exp $ -->
+<!-- $Header: /cvsroot/pgsql/doc/src/sgml/regress.sgml,v 1.16 2001/01/02 05:56:02 tgl Exp $ -->
 
  <chapter id="regress">
   <title id="regress-title">Regression Tests</title>
@@ -248,7 +248,40 @@ SELECT * from iexit;
 </programlisting>
     </para>
    </sect2>
-    
+
+   <sect2>
+    <title>Tuple ordering differences</title>
+      
+    <para>
+You might see differences in which the same tuples are output in a
+different order than what appears in the expected file.  In most cases
+this is not, strictly speaking, a bug.  Most of the regression test
+scripts are not so pedantic as to use an ORDER BY for every single
+SELECT, and so their result tuple orderings are not well-defined
+according to the letter of the SQL spec.  In practice, since we are
+looking at the same queries being executed on the same data by the same
+software, we usually get the same result ordering on all platforms, and
+so the lack of ORDER BY isn't a problem.  Some queries do exhibit
+cross-platform ordering differences, however.
+    </para>
+
+    <para>
+Therefore, if you see an ordering difference, it's not something to
+worry about (unless the query does have an ORDER BY that your result
+is violating).  But please report it anyway, so that we can add an
+ORDER BY to that particular query and thereby eliminate the bogus
+<quote>failure</quote> in future releases.
+    </para>
+
+    <para>
+You might wonder why we don't ORDER all the regress test SELECTs to
+get rid of this issue once and for all.  The reason is that that would
+make the regression tests less useful, not more, since they'd tend
+to exercise query plan types that produce ordered results to the
+exclusion of those that don't.
+    </para>
+   </sect2>
+
    <sect2>
     <title>The <quote>random</quote> test</title>
       
diff --git a/src/test/regress/README b/src/test/regress/README
index 7cfe65b2e55..f687c9aff3e 100644
--- a/src/test/regress/README
+++ b/src/test/regress/README
@@ -166,6 +166,31 @@ statements where these problems occur are the following:
 SELECT * from street;
 SELECT * from iexit;
 
+Tuple ordering differences
+
+You might see differences in which the same tuples are output in a
+different order than what appears in the expected file.  In most cases
+this is not, strictly speaking, a bug.  Most of the regression test
+scripts are not so pedantic as to use an ORDER BY for every single
+SELECT, and so their result tuple orderings are not well-defined
+according to the letter of the SQL spec.  In practice, since we are
+looking at the same queries being executed on the same data by the same
+software, we usually get the same result ordering on all platforms, and
+so the lack of ORDER BY isn't a problem.  Some queries do exhibit
+cross-platform ordering differences, however.
+
+Therefore, if you see an ordering difference, it's not something to
+worry about (unless the query does have an ORDER BY that your result
+is violating).  But please report it anyway, so that we can add an
+ORDER BY to that particular query and thereby eliminate the bogus
+"failure" in future releases.
+
+You might wonder why we don't ORDER all the regress test SELECTs to
+get rid of this issue once and for all.  The reason is that that would
+make the regression tests less useful, not more, since they'd tend
+to exercise query plan types that produce ordered results to the
+exclusion of those that don't.
+
 The "random" test
 
 There is at least one case in the "random" test script that is