Make PL/Python handle domain-type conversions correctly.

Fix PL/Python so that it can handle domains over composite, and so that
it enforces domain constraints correctly in other cases that were not
always done properly before.  Notably, it didn't do arrays of domains
right (oversight in commit c12d570fa), and it failed to enforce domain
constraints when returning a composite type containing a domain field,
and if a transform function is being used for a domain's base type then
it failed to enforce domain constraints on the result.  Also, in many
places it missed checking domain constraints on null values, because
the plpy_typeio code simply wasn't called for Py_None.

Rather than try to band-aid these problems, I made a significant
refactoring of the plpy_typeio logic.  The existing design of recursing
for array and composite members is extended to also treat domains as
containers requiring recursion, and the APIs for the module are cleaned
up and simplified.

The patch also modifies plpy_typeio to rely on the typcache more than
it did before (which was pretty much not at all).  This reduces the
need for repetitive lookups, and lets us get rid of an ad-hoc scheme
for detecting changes in composite types.  I added a couple of small
features to typcache to help with that.

Although some of this is fixing bugs that long predate v11, I don't
think we should risk a back-patch: it's a significant amount of code
churn, and there've been no complaints from the field about the bugs.

Tom Lane, reviewed by Anthony Bykov

Discussion: https://postgr.es/m/24449.1509393613@sss.pgh.pa.us

2 files changed, 34 insertions, 2 deletions

diff --git a/contrib/hstore_plpython/expected/hstore_plpython.out b/contrib/hstore_plpython/expected/hstore_plpython.out
index df49cd5f373..1ab5feea93d 100644
--- a/contrib/hstore_plpython/expected/hstore_plpython.out
+++ b/contrib/hstore_plpython/expected/hstore_plpython.out
@@ -68,12 +68,30 @@ AS $$
 val = [{'a': 1, 'b': 'boo', 'c': None}, {'d': 2}]
 return val
 $$;
- SELECT test2arr();
+SELECT test2arr();
                            test2arr                           
 --------------------------------------------------------------
  {"\"a\"=>\"1\", \"b\"=>\"boo\", \"c\"=>NULL","\"d\"=>\"2\""}
 (1 row)
 
+-- test python -> domain over hstore
+CREATE DOMAIN hstore_foo AS hstore CHECK(VALUE ? 'foo');
+CREATE FUNCTION test2dom(fn text) RETURNS hstore_foo
+LANGUAGE plpythonu
+TRANSFORM FOR TYPE hstore
+AS $$
+return {'a': 1, fn: 'boo', 'c': None}
+$$;
+SELECT test2dom('foo');
+             test2dom              
+-----------------------------------
+ "a"=>"1", "c"=>NULL, "foo"=>"boo"
+(1 row)
+
+SELECT test2dom('bar');  -- fail
+ERROR:  value for domain hstore_foo violates check constraint "hstore_foo_check"
+CONTEXT:  while creating return value
+PL/Python function "test2dom"
 -- test as part of prepare/execute
 CREATE FUNCTION test3() RETURNS void
 LANGUAGE plpythonu
diff --git a/contrib/hstore_plpython/sql/hstore_plpython.sql b/contrib/hstore_plpython/sql/hstore_plpython.sql
index 911bbd67fed..2c54ee6aaad 100644
--- a/contrib/hstore_plpython/sql/hstore_plpython.sql
+++ b/contrib/hstore_plpython/sql/hstore_plpython.sql
@@ -60,7 +60,21 @@ val = [{'a': 1, 'b': 'boo', 'c': None}, {'d': 2}]
 return val
 $$;
 
- SELECT test2arr();
+SELECT test2arr();
+
+
+-- test python -> domain over hstore
+CREATE DOMAIN hstore_foo AS hstore CHECK(VALUE ? 'foo');
+
+CREATE FUNCTION test2dom(fn text) RETURNS hstore_foo
+LANGUAGE plpythonu
+TRANSFORM FOR TYPE hstore
+AS $$
+return {'a': 1, fn: 'boo', 'c': None}
+$$;
+
+SELECT test2dom('foo');
+SELECT test2dom('bar');  -- fail
 
 
 -- test as part of prepare/execute