Bugs item #728330, was opened at 2003-04-27 05:21 Message generated for change (Comment added) made by billbaxter26 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=728330&group_id=5470

Category: Installation Group: Python 2.3 Status: Open Resolution: Later Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Neal Norwitz (nnorwitz) Summary: Don't define _SGAPI on IRIX

Initial Comment: Python release: 2.3b1 Platform: IRIX 6.5 MIPSpro Compilers: Version 7.3.1.2m Commands used: ./configure --prefix=/usr/local_cci/Python-2.3b1t make

The socket module fails to compile. The output from the compiler is attached.

----------------------------------------------------------------------

Comment By: Bill Baxter (billbaxter26) Date: 2004-03-03 11:20

Message: Logged In: YES user_id=892601

Got it down to one error: % > ./python Lib/test/test_socket.py Traceback (most recent call last): File "Lib/test/test_socket.py", line 6, in ? import socket File "/usr/local/spider/lib/python2.3/socket.py", line 44, in ? import _socket ImportError: 210454:./python: rld: Fatal Error: unresolvable symbol in /usr/local/spider/lib/python2.3/lib-dynload/_socketmodule.so: _xpg5_accept

xpg5_accept does appear in any library on our SGI's that are only 1-2 years old. It does appear in libc in very recent revisions of Irix (> 6.5.19?)

Does anyone know how to get Python 2.3 to avoid the use of xpg5_accept?

Thanks

----------------------------------------------------------------------

Comment By: Bill Baxter (billbaxter26) Date: 2004-01-12 13:59

Message: Logged In: YES user_id=892601

Has anyone encountered problems with different versions of IRIX 6.5? I finally got sockets to work in Python2.3.3, but only on the most up-to-date version of IRIX (uname -v = 01100304). On any other machine with a lower version (archetecture doesn't seem to matter), I get messages which all seem to be related to sockets:

152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_vsnprintf 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_recv 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_connect 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_bind 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_getsockopt 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_sendto 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_setsockopt 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_accept 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_recvfrom 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_socket 152272:python2.3: rld: Error: unresolvable symbol in python2.3: _xpg5_getsockname

Upgrading all of our SGIs is not something I have control over.

----------------------------------------------------------------------

Comment By: Maik Hertha (mhertha) Date: 2003-12-22 13:52

Message: Logged In: YES user_id=768223

Sorry if there is something unclear. Redefine means: Set a value which is required to satisfy ifdef-statements in the header files. As you see in the definitions, given in the comment, the value for _NO_XOPEN4 is set as a result of some dependencies automatically defined. A redefinition of _SGIAPI in the socketmodule.c is not sufficient alone. Therefore to satisfy all requirements in the header files, you should set the values for _NO_XOPEN4 and _NO_XOPEN5 also manually. Howto do in socketmodule.c? ... #define _SGIAPI 1 #define _NO_XOPEN4 1 #define _NO_XOPEN5 1 ... If you are familiar with regular expressions, then you will know that _NO_XOPEN[45] is a shorthand for _NO_XOPEN4, _NO_XOPEN5.

Sorry if there was something wrong in my comment.

----------------------------------------------------------------------

Comment By: Bill Baxter (billbaxter26) Date: 2003-12-22 10:49

Message: Logged In: YES user_id=892601

Can someone please clarify mherta's comment from 2003-12-16?

How should we "redefine _NO_XOPEN[45] in the socketmodule.c at the same position like the _SGIAPI redefinition"?

Should those lines of code be added to socketmodule.c, or are they the source of the problem?

Is _NO_XOPEN[45] shorthand for _NO_XOPEN[4] and _NO_XOPEN[5] or are they all different?

Thanks.

----------------------------------------------------------------------

Comment By: Maik Hertha (mhertha) Date: 2003-12-16 12:27

Message: Logged In: YES user_id=768223

I've tested the latest Python 2.3.3c1 to compile. The problem reported is still present. To avoid the error messages, about the incompatible types of several netdb-functions (like inet_pton), you should redefine _NO_XOPEN[45] in the socketmodule.c at the same position like the _SGIAPI redefinition. If there are error messages that types like u_short and u_char are not defined, please add -D_BSD_TYPES in the Makefile (BASECFLAGS is okay). If anybody don't need a executable compiled with MipsPro, gcc from the freeware.sgi.com could be a choice. The problem for this seems in /usr/include/standards.h: #define _NO_XOPEN4 (!defined(_XOPEN_SOURCE) || (defined(_XOPEN_SOURCE) &&
(_XOPEN_SOURCE+0 >= 500))) #define _NO_XOPEN5 (!defined(_XOPEN_SOURCE) || (defined(_XOPEN_SOURCE) &&
(_XOPEN_SOURCE+0 < 500))) As the first option is true (_XOPEN_SOURCE is set to 600 in pyconfig.h) the second option will be false. The constant #define NI_NUMERICHOST 0x00000002 /* return numeric form of hostname */ is enveloped in a clause #ifdef _NO_XOPEN4 && _NO_XOPEN5 (/usr/include/netdb.h) which is always false. In the manual pages there is a example which uses the constant NI_NUMERICHOST and compiles without error message. I'm not sure to cut off the file pyconfig.h from compiling the socketmodule.c?

Any comments?

----------------------------------------------------------------------

Comment By: Hans Nowak (zephyrfalcon) Date: 2003-10-30 22:04

Message: Logged In: YES user_id=173607

My $0.02...

Yesterday I successfully compiled Python 2.3.2, with socket module. I'm using an O2 running IRIX 6.5.11f.

I didn't encounter any of the problems described before, and had no problems with _SGIAPI or anything. The reason it didn't compile was that u_long and u_char were not defined in some places; /usr/include/netinet/tcp.h to be precise, and a Python file (getaddrinfo.c).

I'm using gcc 3.3 (the one downloaded from freeware.sgi.com).

If anyone is interested, drop me a mail at hans at zephyrfalcon.org and I'll put the binary online somewhere.

----------------------------------------------------------------------

Comment By: Bill Baxter (billbaxter26) Date: 2003-10-24 15:59

Message: Logged In: YES user_id=892601

Here's the output from 'gmake', using MIPSpro Compiler 7.30 on IRIX 6.5

Sorry, I don't know how to attach something using the "Add a Comment" box. Is there another way to post?

The patch at www.python.org/2.3.2/patches/sgi-socketmodule.patch was edited into socketmodule.c : #include <netdb.h> #endif

/* GCC 3.2 on Irix 6.5 fails to define this variable at all. Based on the above code, I'd say that the SGI headers are just horked. */ #if defined(__sgi) && !defined(INET_ADDRSTRLEN) #define INET_ADDRSTRLEN 16 #endif

/* Generic includes */ #include <sys/types.h> #include <signal.h>

---------------------------------------------------------------------------

cc -OPT:Olimit=0 -DNDEBUG -n32 -O -I. -I./Include -DPy_BUILD_CORE -c ./Modules/socketmodule.c -o Modules/socketmodule.o cc-1047 cc: WARNING File = ./Modules/socketmodule.c, Line = 195 Macro "_SGIAPI" (declared at line 210 of "/usr/include/standards.h") has an incompatible redefinition.

#define _SGIAPI 1 ^

cc-1020 cc: ERROR File = /usr/include/netdb.h, Line = 198 The identifier "socklen_t" is undefined.

extern int iruserok_sa(const void *, socklen_t, int, const char *, ^

cc-1047 cc: WARNING File = ./Modules/addrinfo.h, Line = 145 Macro "_SS_ALIGNSIZE" (declared at line 246 of "/usr/include/sys/socket.h") has an incompatible redefinition.

#define _SS_ALIGNSIZE (sizeof(PY_LONG_LONG)) ^

cc-1020 cc: ERROR File = ./Modules/socketmodule.c, Line = 822 The identifier "NI_NUMERICHOST" is undefined.

NI_NUMERICHOST); ^

cc-1515 cc: ERROR File = ./Modules/socketmodule.c, Line = 3054 A value of type "int" cannot be assigned to an entity of type "const char *".

retval = inet_ntop(af, packed, ip, sizeof(ip)); ^

3 errors detected in the compilation of "./Modules/socketmodule.c". gmake: *** [Modules/socketmodule.o] Error 2

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh) Date: 2003-10-23 06:25

Message: Logged In: YES user_id=6656

Just to be clear, please "attach" the build rather than pasting it inline.

----------------------------------------------------------------------

Comment By: Anthony Baxter (anthonybaxter) Date: 2003-10-22 22:19

Message: Logged In: YES user_id=29957

Did you try 2.3.2 with the patch from the 2.3.2 build bugs page?

If it failed, please post the build log of the failing bit to this bug.

----------------------------------------------------------------------

Comment By: Bill Baxter (billbaxter26) Date: 2003-10-22 14:26

Message: Logged In: YES user_id=892601

Has there been any progress since August on getting Python-2.3 to compile on IRIX 6.5? I really need to have sockets working to use python.

----------------------------------------------------------------------

Comment By: Maik Hertha (mhertha) Date: 2003-08-14 07:37

Message: Logged In: YES user_id=768223

I have the same problem on IRIX 6.5.19m and MIPSpro 7.3.1.3m on a FUEL 600Mhz. The situation seems a little bit strange on this system. But I have luck and get it compiled. As the current implementation is not clean, my resolution also. Which should be changed to get it work. The manpage for gethostbyaddr_r says: #include <sys/types> #include <sys/socket> #include <netinet/in.h> #include <netdb.h> So I included this in the __sgi-clause in socketmodule.c. Looking in the header-files shows, that some variables are defined if #define _NO_XOPEN4 1 #define _NO_XOPEN5 1 are set, before the includes. This defines some more variables, so that pyconfig.h should be changed. #define HAVE_ADDRINFO 1 #define HAVE_SOCKADDR_STORAGE 1 Sure there are warnings about the redefinition of the _(SGIAPI|NO_XOPEN4|NO_XOPEN5) but it compiles and the tests don't fail. Also my applications run without exception or coredump on FUEL, Octane, Octane 2.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz) Date: 2003-08-01 08:10

Message: Logged In: YES user_id=33168

Ralf, I will try to work on this, but it won't be for a week or two probably. Can you tell me which machines (by name) work and doesn't work. The only IRIX machine I seem to know about it rattler.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2003-08-01 03:34

Message: Logged In: YES user_id=71407

For the records: The Python2.3 (final) socket module still does not compile under IRIX 6.5.20. It only compiles under 6.5.10.

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton) Date: 2003-07-17 14:07

Message: Logged In: YES user_id=31392

I applied a slight variant of Neal's patch and that seemed to fix things. It still seems wrong that we define _SGIAPI, but that will have to wait for 2.3.1.

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton) Date: 2003-07-17 09:54

Message: Logged In: YES user_id=31392

I learned a little about inet_ntop() in the last few minutes. It looks like the size_t argument is an older spelling, and the current POSIX says it should be socklen_t. Regardless, it looks like we're using the new definition when IRIX is actually providing the older definition.

Neal attached a patch that looks like it should clean up the problem. Have you tried it? I think a better solution would be to teach configure how to find inet_ntop() via netdb.h, but that's probably too hard to get done in the new couple of hours.

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton) Date: 2003-07-17 09:42

Message: Logged In: YES user_id=31392

There seem to be two causes of worry. The actual compiler error in your build log and the use of _SGIAPI that Casavanti warns again.

It looks like the compiler problems is because our inet_ntop() prototype is wrong. That is Python says the last argument is a socklen_t, but IRIX and Linux man pages both say size_t. Perhaps size_t and socklen_t are different sizes on IRIX, but not on Linux. What happens if you change it to size_t?

The problematic code would appear to be these lines from socketmodule.c:

#if defined(__sgi)&&_COMPILER_VERSION>700 && !_SGIAPI /* make sure that the reentrant (gethostbyaddr_r etc) functions are declared correctly if compiling with MIPSPro 7.x in ANSI C mode (default) */ #define _SGIAPI 1 #include "netdb.h" #endif

It's the only use the _SGIAPI symbol that Casavant says we shouldn't use. What happens if you remove the use _SGIAPI, i.e. just make it

#if defined(__sgi) && _COMPILER_VERSION > 700 /* make sure that the reentrant (gethostbyaddr_r etc) functions are declared correctly if compiling with MIPSPro 7.x in ANSI C mode (default) */ #include "netdb.h" #endif

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2003-07-16 20:27

Message: Logged In: YES user_id=71407

We have confirmed that the socket module in Python 2.3b1 and 2.3b2 fails to compile under these versions of IRIX:

6.5.10, MIPSpro 7.3.1.2m 6.5.19, MIPSpro 7.3.1.3m 6.5.20, MIPSpro 7.3.1.3m (latest available, to my knowledge)

If Python 2.3 goes out without a working socket module it will be useless for many people in science where IRIX is still found quite frequently. Maybe even worse, we will also no longer be able to advertise Python as a language that "runs everywhere."

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2003-07-15 18:37

Message: Logged In: YES user_id=71407

Date: Tue, 15 Jul 2003 15:06:25 -0500 From: "Brent Casavant" <bcas...@sgi.com> To: "Ralf W. Grosse-Kunstleve" <rw...@yahoo.com> Subject: Re: IRIX & Python

Ralf,

I read through the sourceforge report. I don't have a sourceforge account so I didn't add there, but feel free to distribute my ramblings.

First, they should be very careful about what version of the operating system they are using. Real IPv6 support was only added a few releases ago (6.5.20 I believe, though my memory is hazy). Some of the functions/macros corresponding to IPv6 were introduced prior to that point, but implementation may not have been complete. If they have to play tricks with #define values to get a symbol recongnized it most likely means that SGI didn't intend for that symbol to be used yet.

I suspect the problems they are seeing are because they are using some version of IRIX 6.5.x that includes inet_ntoa() definitions under _SGIAPI, but for which real support was not yet complete. The namespace of symbols that begin with a single underscore and a capital letter is reserved for the OS vendors, and the fact that someone is trying to manually insert a definition of _SGIAPI into the source should be a big red warning flag.

That said, there is a correct method to deal with the new capabilities of IRIX 6.5.x over time. This comes in two parts. A better explanation is in /usr/include/optional_sym.h

First, when a new symbol is added to IRIX, the corresponding header file will always have the following line in it:

#pragma optional some_new_symbol

The presence of this #pragma will cause the linker to not complain if the symbol is missing when the executable is linked. If SGI neglects to do so it is a bug that should be reported to us.

The second part is that calls to these "optional" symbols should be protected by a run-time check to see if the symbol is actually present. Just like this:

#include <optional_sym.h> #include <header_file_for_some_new_symbol.h>

. . .

if (_MIPS_SYMBOL_PRESENT(some_new_symbol)) { /* New function is present, use it */ qux = some_new_symbol(foo, bar, baz); } else { /* New function isn't present, do something else */ qux = builtin_some_new_symbol(foo, bar, baz); }

This ensures that a single executable will be able to function properly (or at least degrade gracefully) on all version of IRIX 6.5.x. In other words you can compile on 6.5.20 and run on 6.5.8, or vice versa.

In particular, this means that compile-time detection of features such as inet_ntoa() is incorrect for IRIX 6.5.x -- these decisions should be made at run-time instead. If the decision is made at compile time the executable may well not execute on an older version of IRIX 6.5.x. Unfortunately GNU configure is not equipped to utilize such advanced binary-compatability mechanisms, so there's no "easy" way to get GNU configure'd software to run across all IRIX 6.5.x releases, short of compiling only on the original non-upgraded version of IRIX 6.5.

Hope that helps, Brent Casavant

On Tue, 15 Jul 2003, Ralf W. Grosse-Kunstleve wrote:

https://sourceforge.net/tracker/? func=detail&aid=728330&group_id=5470&atid=105470

-- Brent Casavant Silicon Graphics, Inc. Where am I? And what am I IRIX O.S. Engineer http://www.sgi.com/ doing in this handbasket? bcas...@sgi.com 44.8562N 93.1355W 860F -- Unknown

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2003-06-30 10:21

Message: Logged In: YES user_id=71407

The socket module still fails to compile with Python 2.3b2, in addition to two other extensions. The full make log is attached.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis) Date: 2003-05-22 02:47

Message: Logged In: YES user_id=21627

I think it would be best if detect presence of inet_aton/inet_pton. Is IPv6 support possible on this system? Then it would be best if that was detected, as well.

If it is not possible to detect aton/pton/ipv6, what is the problem with not using them?

I'd like to have a look at the relevant headers (in particular to understand the duplicate definition of _SGIAPI), indeed.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz) Date: 2003-05-21 19:35

Message: Logged In: YES user_id=33168

Argh!!! This is a nightmare. The attached hack ^h^h^h patch fixes the problem, but ...

_SGIAPI needs to be defined (~192 in socketmodule.c) in order to get the prototypes for inet_aton, inet_pton. But _SGIAPI can't be defined in the standard header files because XOPEN_SOURCE is defined.

Martin, do you have any other suggestions? I don't see how to do this right within configure. If you want I can send you the header file(s).

----------------------------------------------------------------------

Comment By: alexandre gillet (gillet) Date: 2003-05-12 13:41

Message: Logged In: YES user_id=150999

I had the same problem compiling socket on Irix. We did find that on our system ENABLE_IPV6 is not define and all the error are cause by that.

System that do not enable IPV6 are not well supported with the socketmodule.c code. We were able to compile socket after making few change in the code: I am not sure if I did it right but it compile. I will post my patch but I do not see any place to attach my file example: See line 2794,2800

+ #ifdef ENABLE_IPV6 char packed[MAX(sizeof(struct in_addr), sizeof(struct in6_addr))]; + #else + char packed[sizeof(struct in_addr)]; + #endif

I am not sure how to post the patch or file.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2003-05-05 14:44

Message: Logged In: YES user_id=71407

This is not in my area of expertise, but I've tried a couple of (probably stupid) things:

#include <standards.h> in socketmodule.c instead of #define _SGIAPI. This resulted in even more errors.

./configure --disable-ipv6 Same errors as before.

I am lost here. Sorry. But you still have ssh access to rattler.lbl.gov if that helps. Accounts for other Python developers are a possibility.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis) Date: 2003-05-04 08:39

Message: Logged In: YES user_id=21627

Can you analyse these compiler errors in more detail, please? What, in your opinion, is causing these problems, and how would you correct them?

----------------------------------------------------------------------