Documentation
¶
Overview ¶
Package gogopython wraps a Python dynamic library with Go native functions, making embedding of Python in a native Go app relatively easy. (For some definition of easy)
It wraps common Python C API functions needed to manage interpreters and create/modify Python objects. Not all functions are wrapped. Not all features are wrappable in pure Go as in some cases they're C macros.
Since the #1 headache in using an embedding Python interpreter is finding the necessary library, Python home, and paths for packages, gogopython provides a few helper functions to try figuring this out for the user.
Note: Currently only Python 3.12 is supported.
Index ¶
- Constants
- Variables
- func FindPythonHomeAndPaths(exe string) (string, []string, error)
- func LoadLibrary(exe string) error
- func UnicodeToString(unicode PyObjectPtr) (string, error)
- func WCharToString(text WCharPtr) (string, error)
- type EncodingErrors
- type GilType
- type MethodFlags
- type OptimizeLevel
- type PyCFunction
- type PyCodeObjectPtr
- type PyCompilerFlags
- type PyConfig_3_12
- type PyGILState
- type PyInterpreterConfig
- type PyInterpreterStatePtr
- type PyMemAllocator
- type PyMethodDef
- type PyObjectPtr
- type PyPreConfig
- type PySendResult
- type PyStatus
- type PyThreadStatePtr
- type PyTypeObjectPtr
- type PythonLibraryPtr
- type StartToken
- type Type
- type WCharPtr
Constants ¶
const ( PyMemAllocator_NotSet = iota // Don't change allocator (use defaults). PyMemAllocator_Default // Use defaults allocators. PyMemAllocator_Debug // Default with debug hooks. PyMemAllocator_Malloc // Use malloc(3). PyMemAllocator_MallocDebug // Use malloc(3) with debug hooks. PyMemAllocator_PyMalloc // Use Python's pymalloc. PyMemAllocator_PyMallocDebug // Use Python's pymalloc with debug hooks. )
Variables ¶
var ( // Py_DecodeLocale converts a Go string into a Python *wchar_t, optionally // storing some error information in the provided index (if non-nil). Py_DecodeLocale func(s string, index *int) WCharPtr // Py_EncodeLocale converts a Python *wchar_t to a C char*, optionally // storing some error information in the provided index (if non-nil). Py_EncodeLocale func(p WCharPtr, index *int) *byte // PyPreConfig_InitIsolatedConfig pre-initializes the provided Python // interpreter config using "isolated" defaults. PyPreConfig_InitIsolatedConfig func(*PyPreConfig) // PyConfig_InitPythonConfig initializes the provided Python interpreter // config using defaults. PyConfig_InitPythonConfig func(*PyConfig_3_12) // PyConfig_InitIsolatedPythonConfig initializes the provided Python // interpreter config using "isolated" defaults. PyConfig_InitIsolatedPythonConfig func(*PyConfig_3_12) // PyConfig_Clear clears set values in a given PyConfig_3_12. PyConfig_Clear func(*PyConfig_3_12) // Py_FinalizeEx tears down the global Python interpreter state. // // This can deadlock depending on the GIL state. It can also panic. Py_FinalizeEx func() int32 // Py_EndInterpreter tears down a sub-interpreter using the provided // thread state. // // This can panic or deadlock. Be careful! Py_EndInterpreter func(PyThreadStatePtr) // PyGILState_Check reports whether the caller has the GIL. // It returns 1 if true, 0 if false. PyGILState_Check func() int32 // PyGILState_Ensure takes a reference to the GIL. // Caution: this is recursive. PyGILState_Ensure func() PyGILState // PyGILState_Release releases a reference to the GIL. // Caution: this is recursive. PyGILState_Release func(PyGILState) PyEval_AcquireThread func(PyThreadStatePtr) PyEval_ReleaseThread func(PyThreadStatePtr) PyEval_SaveThread func() PyThreadStatePtr PyEval_RestoreThread func(PyThreadStatePtr) PyThreadState_Get func() PyThreadStatePtr PyThreadState_New func(PyInterpreterStatePtr) PyThreadStatePtr PyThreadState_Swap func(PyThreadStatePtr) PyThreadStatePtr PyThreadState_Clear func(PyThreadStatePtr) PyThreadState_Delete func(PyThreadStatePtr) PyThreadState_DeleteCurrent func() PyThreadState_GetInterpreter func(PyThreadStatePtr) PyInterpreterStatePtr PyInterpreterState_Get func() PyInterpreterStatePtr PyInterpreterState_GetID func(PyInterpreterStatePtr) int64 PyInterpreterState_Clear func(PyInterpreterStatePtr) PyInterpreterState_Delete func(PyInterpreterStatePtr) // PyRun_SimpleString evaluates the given Python script in the current // interpreter, returning an exit code based on if there was a Python // exception raised. PyRun_SimpleString func(script string) int32 // PyRun_String evaluates a given Python script in the current interpreter // using the given StartToken mode and globals/locals dicts. // // Globals will be accessible like any global and the script can mutate the // globals mapping using the "globals" keyword in the script. // // Locals will contain any declared local values from the script and is a // simple way to "return" Python data. PyRun_String func(str string, start StartToken, globals, locals PyObjectPtr) PyObjectPtr // Py_CompileString is a simplified form of Py_CompileStringFlags using // default compiler flags. Py_CompileString func(str, filename string, start StartToken) PyCodeObjectPtr // Py_CompileStringFlags is a simplified form of Py_CompileStringExFlags // with optimizations set to UseInterpreterLevel. Py_CompileStringFlags func(str, filename string, start StartToken, flags *PyCompilerFlags) PyCodeObjectPtr // Py_CompileStringExFlags compiles the Python script in str and returns // the compiled Python code object. The filename is used to populate the // __file__ information for tracebacks and exception messages. // // Returns NullPyCodeObjectPtr on error. Py_CompileStringExFlags func(str, filename string, start StartToken, flags *PyCompilerFlags, optimize OptimizeLevel) PyCodeObjectPtr PyEval_EvalCode func(co PyCodeObjectPtr, globals, locals PyObjectPtr) PyObjectPtr PyModule_New func(string) PyObjectPtr PyModule_GetDict func(ptr PyObjectPtr) PyObjectPtr PyModule_AddObjectRef func(module PyObjectPtr, name string, item PyObjectPtr) int32 PyImport_AddModule func(name string) PyObjectPtr PyImport_ExecCodeModule func(name string, code PyCodeObjectPtr) PyObjectPtr PyImport_GetModuleDict func() PyObjectPtr PyImport_ImportModule func(name string) PyObjectPtr PyImport_ImportModuleLevel func(name string, globals, locals, fromList PyObjectPtr, level int32) PyObjectPtr PyImport_ExecCodeModuleWithPathnames func(name string, code PyCodeObjectPtr, pathname, cPathname string) PyObjectPtr PyCFunction_NewEx func(def *PyMethodDef, self, module PyObjectPtr) PyObjectPtr PyBool_FromLong func(int64) PyObjectPtr PyLong_AsLong func(PyObjectPtr) int64 PyLong_AsLongAndOverflow func(PyObjectPtr, *int64) int64 PyLong_AsUnsignedLong func(PyObjectPtr) uint64 PyLong_FromLong func(int64) PyObjectPtr PyLong_FromUnsignedLong func(uint64) PyObjectPtr PyLong_FromLongLong func(int64) PyObjectPtr PyLong_FromUnsignedLongLong func(uint64) PyObjectPtr PyFloat_AsDouble func(PyObjectPtr) float64 PyFloat_FromDouble func(float64) PyObjectPtr PyTuple_New func(int64) PyObjectPtr PyTuple_GetItem func(tuple PyObjectPtr, pos int64) PyObjectPtr PyTuple_SetItem func(tuple PyObjectPtr, pos int64, item PyObjectPtr) int32 PyTuple_Size func(tuple PyObjectPtr) int64 PyList_New func(PyObjectPtr) int32 PyList_Size func(PyObjectPtr) int64 PyList_GetItem func(PyObjectPtr, int64) PyObjectPtr PyList_SetItem func(list PyObjectPtr, index int, item PyObjectPtr) int32 PyList_Append func(list, item PyObjectPtr) int32 PyList_Insert func(list PyObjectPtr, index int, item PyObjectPtr) int32 PyDict_New func() PyObjectPtr PyDictProxy_New func(mapping PyObjectPtr) PyObjectPtr PyDict_Clear func(PyObjectPtr) PyDict_SetItem func(dict, key, val PyObjectPtr) int32 PyDict_SetItemString func(dict PyObjectPtr, key string, val PyObjectPtr) int64 PyDict_GetItem func(dict, key PyObjectPtr) PyObjectPtr PyDict_GetItemString func(dict PyObjectPtr, key string) PyObjectPtr PyDict_Keys func(dict PyObjectPtr) PyObjectPtr PyDict_Values func(dict PyObjectPtr) PyObjectPtr PyDict_Size func(dict PyObjectPtr) int64 PyIter_Check func(iter PyObjectPtr) int32 PyIter_Next func(iter PyObjectPtr) PyObjectPtr PyIter_Send func(iter, arg PyObjectPtr, result *PyObjectPtr) PySendResult PyFunction_GetCode func(fn PyObjectPtr) PyCodeObjectPtr PyObject_Call func(callable, args, kwargs PyObjectPtr) PyObjectPtr PyObject_CallNoArgs func(callable PyObjectPtr) PyObjectPtr PyObject_CallOneArg func(callable, args PyObjectPtr) PyObjectPtr PyObject_CallMethodNoArgs func(obj, name PyObjectPtr) PyObjectPtr PyObject_CallMethodOneArg func(obj, name, arg PyObjectPtr) PyObjectPtr PyObject_CallObject func(callable, args PyObjectPtr) PyObjectPtr PyObject_IsInstance func(inst, cls PyObjectPtr) int32 PyObject_GetAttrString func(obj PyObjectPtr, name string) PyObjectPtr PySet_New func(iterable PyObjectPtr) PyObjectPtr PyFrozenSet_New func(iterable PyObjectPtr) PyObjectPtr PySet_Size func(PyObjectPtr) int64 PySet_Contains func(set, key PyObjectPtr) int32 PySet_Add func(set, key PyObjectPtr) int32 PySet_Discard func(set, key PyObjectPtr) int32 PySet_Pop func(set, key PyObjectPtr) PyObjectPtr PySet_Clear func(set PyObjectPtr) int32 PyBytes_FromString func(string) PyObjectPtr PyBytes_FromStringAndSize func(*byte, int64) PyObjectPtr PyByteArray_FromStringAndSize func(*byte, int64) PyObjectPtr PyBytes_AsString func(PyObjectPtr) *byte PyBytes_Size func(PyObjectPtr) int64 PyUnicode_FromString func(string) PyObjectPtr PyUnicode_AsEncodedString func(unicode PyObjectPtr, encoding string, errors EncodingErrors) PyObjectPtr PyUnicode_AsWideCharString func(PyObjectPtr, *int) WCharPtr PyUnicode_DecodeFSDefault func(string) PyObjectPtr PyUnicode_EncodeFSDefault func(PyObjectPtr) PyObjectPtr Py_DecRef func(PyObjectPtr) Py_IncRef func(PyObjectPtr) PyErr_Clear func() PyErr_Print func() PyMem_Free func(*byte) PyObject_Type func(PyObjectPtr) PyTypeObjectPtr PyType_GetFlags func(PyTypeObjectPtr) uint64 )
var ( Py_PreInitialize func(*PyPreConfig) PyStatus PyConfig_SetBytesString func(*PyConfig_3_12, *WCharPtr, string) PyStatus Py_InitializeFromConfig func(*PyConfig_3_12) PyStatus Py_NewInterpreterFromConfig func(state *PyThreadStatePtr, c *PyInterpreterConfig) PyStatus )
Our problem children. These all return PyStatus, a struct. These need special handling to work on certain platforms like Linux due to how purego is currently written.
Functions ¶
func FindPythonHomeAndPaths ¶ added in v0.5.0
FindPythonHomeAndPaths uses the provided Python executable to discovery the Python Home and path settings.
func LoadLibrary ¶ added in v0.17.0
LoadLibrary attempts to load and wrap the appropriate dynamic library for embedding Python, given a particular Python binary.
Currently, assumes the provided binary is for Python 3.12.
func UnicodeToString ¶ added in v0.8.0
func UnicodeToString(unicode PyObjectPtr) (string, error)
UnicodeToString converts a Python Unicode object (i.e. a Python string) to a Go string.
Note: this currently involves a lot of data copying out from the Python interpreter. It's far from optimized.
func WCharToString ¶ added in v0.8.0
WCharToString copies out a Python *wchar_t to a Go string.
Types ¶
type EncodingErrors ¶ added in v0.17.0
type EncodingErrors = string
const ( Strict EncodingErrors = "strict" Ignore EncodingErrors = "ignore" Replace EncodingErrors = "replace" XmlCharRefReplace EncodingErrors = "xmlcharrefreplace" BackslashReplace EncodingErrors = "backslashreplace" )
type MethodFlags ¶ added in v0.9.0
type MethodFlags int32
MethodFlags bits indicate how the method call is constructed.
const ( MethodVarArgs MethodFlags = 0x01 MethodKeywords MethodFlags = 0x02 MethodNoArgs MethodFlags = 0x04 MethodFastCall MethodFlags = 0x80 )
const ( // PyCFunctionDefault has the C function signature: // PyObject *PyCFunction(PyObject *self, PyObject *args) PyCFunctionDefault MethodFlags = MethodVarArgs // PyCFunctionWithKeywords has the C function signature: // PyObject *fn(PyObject *self, PyObject *args, PyObject *kwargs) PyCFunctionWithKeywords MethodFlags = MethodVarArgs | MethodKeywords // PyCFunctionFast has the C function signature: // PyObject *fn(PyObject *self, PyObject *const *args, Py_ssize_t nargs) PyCFunctionFast MethodFlags = MethodFastCall )
type OptimizeLevel ¶ added in v0.6.0
type OptimizeLevel = int32
const ( UseInterpreterLevel OptimizeLevel = -1 // Uses whatever the interpreter was built with. NoOptimization OptimizeLevel = 0 // No optimization, __debug__ is True. RemoveDebugsAndAsserts OptimizeLevel = 1 // __debug__ is False, no asserts. RemoveDebugsAssertsAndDocstrings OptimizeLevel = 2 // __debug__ is False, no asserts, no docstrings. )
type PyCFunction ¶ added in v0.9.0
type PyCFunction = uintptr
PyCFunction points to a C function implementation.
type PyCodeObjectPtr ¶ added in v0.6.0
type PyCodeObjectPtr PyObjectPtr
PyCodeObjectPtr is a pointer to an underlying Python code object.
const NullPyCodeObjectPtr PyCodeObjectPtr = 0
NullPyCodeObjectPtr represents a NULL pointer to a Python code object.
type PyCompilerFlags ¶ added in v0.6.0
type PyConfig_3_12 ¶
type PyConfig_3_12 struct {
ConfigInit int32
Isolated int32
UseEnvironment int32
DevMode int32
InstallSignalHandlers int32
UseHashSeed int32
HashSeed uint64
FaultHandler int32
TraceMalloc int32
PerfProfiling int32
ImportTime int32
CodeDebugRanges int32
ShowRefCount int32
DumpRefs int32
DumpRefsFile WCharPtr
MallocStats int32
FilesystemEncoding WCharPtr
FilesystemErrors WCharPtr
PycachePrefix WCharPtr
ParseArgv int32
OrigArgv pyWideStringList
Argv pyWideStringList
XOptions pyWideStringList
WarnOptions pyWideStringList
SiteImport int32
BytesWarning int32
WarnDefaultEncoding int32
Inspect int32
Interactive int32
OptimizationLevel int32
ParserDebug int32
WriteBytecode int32
Verbose int32
Quiet int32
UserSiteDirectory int32
ConfigureCStdio int32
BufferedStdio int32
StdioEncodings WCharPtr
StdioErrors WCharPtr
// LegacyWindowsStdio int32 // if windows
CheckHashPycsMode WCharPtr
UseFrozenModules int32
SafePath int32
IntMaxStrDigits int32
/* Path configuration inputs */
PathConfigWarnings int32
ProgramName WCharPtr
PythonPathEnv WCharPtr
Home WCharPtr
PlatLibDir WCharPtr
/* Path configuration outputs */
ModuleSearchPathsSet int32
ModuleSearchPaths pyWideStringList
StdlibDir *byte
Executable *byte
BaseExecutable *byte
Prefix *byte
BasePrefix *byte
ExecPrefix *byte
BaseExecPrefix *byte
/* Parameter only used by Py_Main */
SkipSourceFirstLine int32
RunCommand *byte
RunModule *byte
RunFilename *byte
/* Set by Py_Main */
SysPath0 *byte
/* Private Fields */
InstallImportLib int32
InitMain int32
IsPythonBuild int32
}
PyConfig_3_12 configures a Python 3.12 interpreter.
This is a version-dependent structure, unfortunately. We need this because it's the stable way of configuring the Home and Path (PythonPathEnv).
Sadly this is also dependent on platform (Windows vs. not-Windows) and some compile time decisions for the Python implementation (e.g. debug, stats).
Ultimately, this should be made private and the configuration complexity hidden from the programmer.
type PyGILState ¶
type PyGILState int32
PyGILState is used sort as a cookie in the ensure/release function calls.
type PyInterpreterConfig ¶
type PyInterpreterConfig struct {
// Whether to share the main interpreters object allocator state.
//
// If this is 0, you must set CheckMultiInterpExtensions to 1.
// If this is 1, you must set Gil to OwnGil.
UseMainObMalloc int32
// Whether to allow using Python's os.fork function.
// Note: this doesn't block exec syscalls and subprocess module will still work.
AllowFork int32
// Whether to allow using Python's os.exec* functions.
// Note: this doesn't block exec syscalls and subprocess module will still work.
AllowExec int32
// Whether to allow creating Python threads using the threading module.
AllowThreads int32
// Whether to allow creating Python daemon threads.
AllowDaemonThreads int32
// If 1, require multi-phase (non-legacy) extension modules. Must be 1 if you
// enable UseMainObMalloc.
CheckMultiInterpExtensions int32
// The GIL mode for this sub-interpreter.
Gil GilType
}
PyInterpreterConfig defines settings for a sub-interpreter. All int32 values are really booleans, so 0 = false, 1 = true. (Non-zero may also be equivalent to true, but I'm not sure!)
type PyInterpreterStatePtr ¶
type PyInterpreterStatePtr uintptr
PyInterpreterStatePtr is a pointer to a Python InterpreterState.
const NullInterpreterState PyInterpreterStatePtr = 0
NullInterpreterState is a NULL version of a Python InterpreterState.
type PyMemAllocator ¶ added in v0.5.0
type PyMemAllocator = int32
type PyMethodDef ¶ added in v0.9.0
type PyMethodDef struct {
Name *byte // Name is a C string identifying the method.
Method PyCFunction // Method points to the C function to be called.
Flags MethodFlags // Flags is the bit-wise configuration of how the method is invoked.
Docstring *byte // Docstring is a C string describing documentation for the method.
}
PyMethodDef describes a C function made callable from Python.
type PyObjectPtr ¶
type PyObjectPtr uintptr
PyObjectPtr is a pointer to an underlying PyObject instance.
const NullPyObjectPtr PyObjectPtr = 0
NullPyObjectPtr represents a NULL pointer to a Python PyObject
func NewFunction ¶ added in v0.9.0
func NewFunction(name string, self PyObjectPtr, fn func(self, tuple PyObjectPtr) PyObjectPtr) PyObjectPtr
NewFunction creates a new Python function object, with the given name, that calls the provided Go func.
type PyPreConfig ¶
type PySendResult ¶ added in v0.11.0
type PySendResult int32
const ( PyGen_Return PySendResult = 0 PyGen_Error PySendResult = -1 PyGen_Next PySendResult = 1 )
type PyStatus ¶
PyStatus is returned by some Python C API calls.
This is the biggest headache of this whole thing. A few functions return this struct directly instead of either via a pointer or by reference in the function args. It creates a nightmare to deal with the various ABI logic for how structs get returned that don't fit into a cpu register width.
If someone has a time machine, please go back and tell Guido not to do this. Please.
type PyThreadStatePtr ¶
type PyThreadStatePtr uintptr
PyThreadStatePtr is a pointer to a Python ThreadState.
const NullThreadState PyThreadStatePtr = 0
NullThreadState is a NULL version of a Python ThreadState.
type PyTypeObjectPtr ¶ added in v0.3.0
type PyTypeObjectPtr PyObjectPtr
PyTypeObjectPtr is a pointer to an underlying PyTypeObject instance.
const NullPyTypeObjectPtr PyTypeObjectPtr = 0
NullPyTypeObjectPtr represents a NULL pointer to a Python PyTypeObject
type PythonLibraryPtr ¶
type PythonLibraryPtr = uintptr
PythonLibraryPtr is a pointer to a Python dynamic library state.
type StartToken ¶
type StartToken = int32
StartToken (confusingly named) is used to dictate to the Python interpreter and compiler how to evaluate a Python script.
const ( PySingleInput StartToken = 256 // Used for single statements. PyFileInput StartToken = 257 // Used for modules (many statements). PyEvalInput StartToken = 258 // Used for expressions(?). PyFuncTypeInput StartToken = 345 // No idea what this is! )
type Type ¶ added in v0.3.0
type Type uint64
Type describes the native Python type of PyObject.
const ( Long Type = (1 << 24) // Python long. List Type = (1 << 25) // Python list. Tuple Type = (1 << 26) // Python tuple. Bytes Type = (1 << 27) // Python bytes (not bytearray). String Type = (1 << 28) // Python Unicode string. Dict Type = (1 << 29) // Python dictionary. None Type = 0 // The Python "None" type. Float Type = 1 // Python float. Set Type = 2 // Python set. Function Type = 3 // Python function. Generator Type = 4 // Python generator. Module Type = 5 // Python module. Bool Type = 6 Unknown Type = 0xffffffff // We have no idea what the type is... )
func BaseType ¶ added in v0.8.0
func BaseType(obj PyObjectPtr) Type
BaseType identifies the Python base type from a Python *PyObject.
This uses a heuristic based on inspecting some internal object flags as most of the Python C API for type inspection is written in macros.
See https://docs.python.org/3/c-api/type.html#c.PyType_GetFlags if curious about the flags.