Iterators, other Python C API improvements

closes #72
ref #69 - this resolves the "UICollection" (not "UIEntityCollection", perhaps renamed since the issue opened) and "UIEntityCollection" portion. The Grid point based iterators were not updated.
**RPATH updates**
Will this RPATH setting allow McRogueFace to execute using its included "lib" subdirectory after being unzipped on a new computer?

The change from "./lib" to "$ORIGIN/./lib" improves portability. The $ORIGIN token is a special Linux/Unix convention that refers to the directory containing the executable itself. This makes the path relative to the executable's location rather than the current working directory, which means McRogueFace will correctly find its libraries in the lib subdirectory regardless of where it's run from after being unzipped on a new computer.

**New standard object initialization**
PyColor, PyVector
  - Fixed all 15 PyTypeObject definitions to use proper designated initializer syntax
  - Replaced PyType_GenericAlloc usage in PyColor.cpp and PyVector.cpp
  - Updated PyObject_New usage in UIEntity.cpp
  - All object creation now uses module-based type lookups instead of static references
  - Created centralized utilities in PyObjectUtils.h

**RAII Wrappers**
automatic reference counting via C++ object lifecycle
  - Created PyRAII.h with PyObjectRef and PyTypeRef classes
  - These provide automatic reference counting management
  - Updated PyColor::from_arg() to demonstrate RAII usage
  - Prevents memory leaks and reference counting errors

**Python object base in type defs:**
`.ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0}`
PyColor, PyTexture, PyVector, UICaption, UICollection, UIEntity, UIFrame, UIGrid

**convertDrawableToPython**
replace crazy macro to detect the correct Python type of a UIDrawable instance

  - Removed the problematic macro from UIDrawable.h
  - Created template-based functions in PyObjectUtils.h
  - Updated UICollection.cpp to use local helper function
  - The new approach is cleaner, more debuggable, and avoids static type references

**Iterator fixes**
tp_iter on UICollection, UIGrid, UIGridPoint, UISprite
UIGrid logic improved, standard

**List vs Vector usage analysis**
there are different use cases that weren't standardized:
  - UICollection (for Frame children) uses std::vector<std::shared_ptr<UIDrawable>>
  - UIEntityCollection (for Grid entities) uses std::list<std::shared_ptr<UIEntity>>

The rationale is currently connected to frequency of expected changes.
* A "UICollection" is likely either all visible or not; it's also likely to be created once and have a static set of contents. They should be contiguous in memory in hopes that this helps rendering speed.
* A "UIEntityCollection" is expected to be rendered as a subset within the visible rectangle of the UIGrid. Scrolling the grid or gameplay logic is likely to frequently create and destroy entities. In general I expect Entity collections to have a much higher common size than UICollections. For these reasons I've made them Lists in hopes that they never have to be reallocated or moved during a frame.

CMakeLists.txt

@@ -46,7 +46,7 @@ endif()
 
 # Add the directory where the linker should look for the libraries
 #link_directories(${CMAKE_SOURCE_DIR}/deps_linux)
-link_directories(${CMAKE_SOURCE_DIR}/lib)
+link_directories(${CMAKE_SOURCE_DIR}/__lib)
 
 # Define the executable target before linking libraries
 add_executable(mcrogueface ${SOURCES})
@@ -67,9 +67,9 @@ add_custom_command(TARGET mcrogueface POST_BUILD
 # Copy Python standard library to build directory
 add_custom_command(TARGET mcrogueface POST_BUILD
     COMMAND ${CMAKE_COMMAND} -E copy_directory
-    ${CMAKE_SOURCE_DIR}/lib $<TARGET_FILE_DIR:mcrogueface>/lib)
+    ${CMAKE_SOURCE_DIR}/__lib $<TARGET_FILE_DIR:mcrogueface>/lib)
 
 # rpath for including shared libraries
 set_target_properties(mcrogueface PROPERTIES
-                      INSTALL_RPATH "./lib")
+                      INSTALL_RPATH "$ORIGIN/./lib")
 

src/McRFPy_API.cpp

@@ -84,9 +84,15 @@ PyObject* PyInit_mcrfpy()
     auto t = pytypes[i];
     while (t != nullptr)
     {
-        /*std::cout << */ PyType_Ready(t); /*<< std::endl; */
+        std::cout << "Registering type: " << t->tp_name << std::endl;
+        if (PyType_Ready(t) < 0) {
+            std::cout << "ERROR: PyType_Ready failed for " << t->tp_name << std::endl;
+            return NULL;
+        }
+        std::cout << "  tp_alloc after PyType_Ready: " << (void*)t->tp_alloc << std::endl;
         PyModule_AddType(m, t);
-        t = pytypes[i++];
+        i++;
+        t = pytypes[i];
     }
 
     // Add default_font and default_texture to module

src/PyColor.cpp

@@ -1,5 +1,7 @@
 #include "PyColor.h"
 #include "McRFPy_API.h"
+#include "PyObjectUtils.h"
+#include "PyRAII.h"
 
 PyGetSetDef PyColor::getsetters[] = {
     {"r", (getter)PyColor::get_member, (setter)PyColor::set_member, "Red component",   (void*)0},
@@ -14,11 +16,16 @@ PyColor::PyColor(sf::Color target)
 
 PyObject* PyColor::pyObject()
 {
-    PyObject* obj = PyType_GenericAlloc(&mcrfpydef::PyColorType, 0);
+    PyTypeObject* type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Color");
-    Py_INCREF(obj);
+    if (!type) return nullptr;
-    PyColorObject* self = (PyColorObject*)obj;
+    
-    self->data = data;
+    PyColorObject* obj = (PyColorObject*)type->tp_alloc(type, 0);
-    return obj;
+    Py_DECREF(type);
+    
+    if (obj) {
+        obj->data = data;
+    }
+    return (PyObject*)obj;
 }
 
 sf::Color PyColor::fromPy(PyObject* obj)
@@ -138,13 +145,30 @@ int PyColor::set_member(PyObject* obj, PyObject* value, void* closure)
 
 PyColorObject* PyColor::from_arg(PyObject* args)
 {
-    auto type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Color");
+    // Use RAII for type reference management
-    if (PyObject_IsInstance(args, (PyObject*)type)) return (PyColorObject*)args;
+    PyRAII::PyTypeRef type("Color", McRFPy_API::mcrf_module);
-    auto obj = (PyColorObject*)type->tp_alloc(type, 0);
+    if (!type) {
-    int err = init(obj, args, NULL);
-    if (err) {
-        Py_DECREF(obj);
         return NULL;
     }
-    return obj;
+    
+    // Check if args is already a Color instance
+    if (PyObject_IsInstance(args, (PyObject*)type.get())) {
+        return (PyColorObject*)args;
+    }
+    
+    // Create new Color object using RAII
+    PyRAII::PyObjectRef obj(type->tp_alloc(type.get(), 0), true);
+    if (!obj) {
+        return NULL;
+    }
+    
+    // Initialize the object
+    int err = init((PyColorObject*)obj.get(), args, NULL);
+    if (err) {
+        // obj will be automatically cleaned up when it goes out of scope
+        return NULL;
+    }
+    
+    // Release ownership and return
+    return (PyColorObject*)obj.release();
 }

src/PyColor.h

@@ -34,6 +34,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyColorType = {
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Color",
         .tp_basicsize = sizeof(PyColorObject),
         .tp_itemsize = 0,

src/PyFont.h

@@ -25,6 +25,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyFontType = {
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Font",
         .tp_basicsize = sizeof(PyFontObject),
         .tp_itemsize = 0,

src/PyObjectUtils.h

@@ -0,0 +1,76 @@
+#pragma once
+#include "Common.h"
+#include "Python.h"
+#include "McRFPy_API.h"
+#include "PyRAII.h"
+
+namespace PyObjectUtils {
+    
+    // Template for getting Python type object from module
+    template<typename T>
+    PyTypeObject* getPythonType(const char* typeName) {
+        PyTypeObject* type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, typeName);
+        if (!type) {
+            PyErr_Format(PyExc_RuntimeError, "Could not find %s type in module", typeName);
+        }
+        return type;
+    }
+    
+    // Generic function to create a Python object of given type
+    inline PyObject* createPyObjectGeneric(const char* typeName) {
+        PyTypeObject* type = getPythonType<void>(typeName);
+        if (!type) return nullptr;
+        
+        PyObject* obj = type->tp_alloc(type, 0);
+        Py_DECREF(type);
+        
+        return obj;
+    }
+    
+    // Helper function to allocate and initialize a Python object with data
+    template<typename PyObjType, typename DataType>
+    PyObject* createPyObjectWithData(const char* typeName, DataType data) {
+        PyTypeObject* type = getPythonType<void>(typeName);
+        if (!type) return nullptr;
+        
+        PyObjType* obj = (PyObjType*)type->tp_alloc(type, 0);
+        Py_DECREF(type);
+        
+        if (obj) {
+            obj->data = data;
+        }
+        return (PyObject*)obj;
+    }
+    
+    // Function to convert UIDrawable to appropriate Python object
+    // This is moved to UICollection.cpp to avoid circular dependencies
+    
+    // RAII-based object creation example
+    inline PyObject* createPyObjectGenericRAII(const char* typeName) {
+        PyRAII::PyTypeRef type(typeName, McRFPy_API::mcrf_module);
+        if (!type) {
+            PyErr_Format(PyExc_RuntimeError, "Could not find %s type in module", typeName);
+            return nullptr;
+        }
+        
+        PyObject* obj = type->tp_alloc(type.get(), 0);
+        // Return the new reference (caller owns it)
+        return obj;
+    }
+    
+    // Example of using PyObjectRef for safer reference management
+    template<typename PyObjType, typename DataType>
+    PyObject* createPyObjectWithDataRAII(const char* typeName, DataType data) {
+        PyRAII::PyObjectRef obj = PyRAII::createObject<PyObjType>(typeName, McRFPy_API::mcrf_module);
+        if (!obj) {
+            PyErr_Format(PyExc_RuntimeError, "Could not create %s object", typeName);
+            return nullptr;
+        }
+        
+        // Access the object through the RAII wrapper
+        ((PyObjType*)obj.get())->data = data;
+        
+        // Release ownership to return to Python
+        return obj.release();
+    }
+}

src/PyRAII.h

@@ -0,0 +1,138 @@
+#pragma once
+#include "Python.h"
+#include <utility>
+
+namespace PyRAII {
+    
+    // RAII wrapper for PyObject* that automatically manages reference counting
+    class PyObjectRef {
+    private:
+        PyObject* ptr;
+        
+    public:
+        // Constructors
+        PyObjectRef() : ptr(nullptr) {}
+        
+        explicit PyObjectRef(PyObject* p, bool steal_ref = false) : ptr(p) {
+            if (ptr && !steal_ref) {
+                Py_INCREF(ptr);
+            }
+        }
+        
+        // Copy constructor
+        PyObjectRef(const PyObjectRef& other) : ptr(other.ptr) {
+            if (ptr) {
+                Py_INCREF(ptr);
+            }
+        }
+        
+        // Move constructor  
+        PyObjectRef(PyObjectRef&& other) noexcept : ptr(other.ptr) {
+            other.ptr = nullptr;
+        }
+        
+        // Destructor
+        ~PyObjectRef() {
+            Py_XDECREF(ptr);
+        }
+        
+        // Copy assignment
+        PyObjectRef& operator=(const PyObjectRef& other) {
+            if (this != &other) {
+                Py_XDECREF(ptr);
+                ptr = other.ptr;
+                if (ptr) {
+                    Py_INCREF(ptr);
+                }
+            }
+            return *this;
+        }
+        
+        // Move assignment
+        PyObjectRef& operator=(PyObjectRef&& other) noexcept {
+            if (this != &other) {
+                Py_XDECREF(ptr);
+                ptr = other.ptr;
+                other.ptr = nullptr;
+            }
+            return *this;
+        }
+        
+        // Access operators
+        PyObject* get() const { return ptr; }
+        PyObject* operator->() const { return ptr; }
+        PyObject& operator*() const { return *ptr; }
+        operator bool() const { return ptr != nullptr; }
+        
+        // Release ownership (for returning to Python)
+        PyObject* release() {
+            PyObject* temp = ptr;
+            ptr = nullptr;
+            return temp;
+        }
+        
+        // Reset with new pointer
+        void reset(PyObject* p = nullptr, bool steal_ref = false) {
+            if (p != ptr) {
+                Py_XDECREF(ptr);
+                ptr = p;
+                if (ptr && !steal_ref) {
+                    Py_INCREF(ptr);
+                }
+            }
+        }
+    };
+    
+    // Helper class for managing PyTypeObject* references from module lookups
+    class PyTypeRef {
+    private:
+        PyTypeObject* type;
+        
+    public:
+        PyTypeRef() : type(nullptr) {}
+        
+        explicit PyTypeRef(const char* typeName, PyObject* module) {
+            type = (PyTypeObject*)PyObject_GetAttrString(module, typeName);
+            // GetAttrString returns a new reference, so we own it
+        }
+        
+        ~PyTypeRef() {
+            Py_XDECREF((PyObject*)type);
+        }
+        
+        // Delete copy operations to prevent accidental reference issues
+        PyTypeRef(const PyTypeRef&) = delete;
+        PyTypeRef& operator=(const PyTypeRef&) = delete;
+        
+        // Allow move operations
+        PyTypeRef(PyTypeRef&& other) noexcept : type(other.type) {
+            other.type = nullptr;
+        }
+        
+        PyTypeRef& operator=(PyTypeRef&& other) noexcept {
+            if (this != &other) {
+                Py_XDECREF((PyObject*)type);
+                type = other.type;
+                other.type = nullptr;
+            }
+            return *this;
+        }
+        
+        PyTypeObject* get() const { return type; }
+        PyTypeObject* operator->() const { return type; }
+        operator bool() const { return type != nullptr; }
+    };
+    
+    // Convenience function to create a new object with RAII
+    template<typename PyObjType>
+    PyObjectRef createObject(const char* typeName, PyObject* module) {
+        PyTypeRef type(typeName, module);
+        if (!type) {
+            return PyObjectRef();
+        }
+        
+        PyObject* obj = type->tp_alloc(type.get(), 0);
+        // tp_alloc returns a new reference, so we steal it
+        return PyObjectRef(obj, true);
+    }
+}

src/PyTexture.h

@@ -29,6 +29,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyTextureType = {
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Texture",
         .tp_basicsize = sizeof(PyTextureObject),
         .tp_itemsize = 0,

src/PyVector.cpp

@@ -1,4 +1,5 @@
 #include "PyVector.h"
+#include "PyObjectUtils.h"
 
 PyGetSetDef PyVector::getsetters[] = {
     {"x", (getter)PyVector::get_member, (setter)PyVector::set_member, "X/horizontal component",   (void*)0},
@@ -11,11 +12,16 @@ PyVector::PyVector(sf::Vector2f target)
 
 PyObject* PyVector::pyObject()
 {
-    PyObject* obj = PyType_GenericAlloc(&mcrfpydef::PyVectorType, 0);
+    PyTypeObject* type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Vector");
-    Py_INCREF(obj);
+    if (!type) return nullptr;
-    PyVectorObject* self = (PyVectorObject*)obj;
+    
-    self->data = data;
+    PyVectorObject* obj = (PyVectorObject*)type->tp_alloc(type, 0);
-    return obj;
+    Py_DECREF(type);
+    
+    if (obj) {
+        obj->data = data;
+    }
+    return (PyObject*)obj;
 }
 
 sf::Vector2f PyVector::fromPy(PyObject* obj)

src/PyVector.h

@@ -30,6 +30,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyVectorType = {
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Vector",
         .tp_basicsize = sizeof(PyVectorObject),
         .tp_itemsize = 0,

src/UICaption.h

@@ -27,6 +27,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUICaptionType = {
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Caption",
         .tp_basicsize = sizeof(PyUICaptionObject),
         .tp_itemsize = 0,

src/UICollection.cpp

@@ -5,9 +5,76 @@
 #include "UISprite.h"
 #include "UIGrid.h"
 #include "McRFPy_API.h"
+#include "PyObjectUtils.h"
 
 using namespace mcrfpydef;
 
+// Local helper function to convert UIDrawable to appropriate Python object
+static PyObject* convertDrawableToPython(std::shared_ptr<UIDrawable> drawable) {
+    if (!drawable) {
+        Py_RETURN_NONE;
+    }
+    
+    PyTypeObject* type = nullptr;
+    PyObject* obj = nullptr;
+    
+    switch (drawable->derived_type()) {
+        case PyObjectsEnum::UIFRAME:
+        {
+            type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Frame");
+            if (!type) return nullptr;
+            auto pyObj = (PyUIFrameObject*)type->tp_alloc(type, 0);
+            if (pyObj) {
+                pyObj->data = std::static_pointer_cast<UIFrame>(drawable);
+            }
+            obj = (PyObject*)pyObj;
+            break;
+        }
+        case PyObjectsEnum::UICAPTION:
+        {
+            type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Caption");
+            if (!type) return nullptr;
+            auto pyObj = (PyUICaptionObject*)type->tp_alloc(type, 0);
+            if (pyObj) {
+                pyObj->data = std::static_pointer_cast<UICaption>(drawable);
+                pyObj->font = nullptr;
+            }
+            obj = (PyObject*)pyObj;
+            break;
+        }
+        case PyObjectsEnum::UISPRITE:
+        {
+            type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Sprite");
+            if (!type) return nullptr;
+            auto pyObj = (PyUISpriteObject*)type->tp_alloc(type, 0);
+            if (pyObj) {
+                pyObj->data = std::static_pointer_cast<UISprite>(drawable);
+            }
+            obj = (PyObject*)pyObj;
+            break;
+        }
+        case PyObjectsEnum::UIGRID:
+        {
+            type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Grid");
+            if (!type) return nullptr;
+            auto pyObj = (PyUIGridObject*)type->tp_alloc(type, 0);
+            if (pyObj) {
+                pyObj->data = std::static_pointer_cast<UIGrid>(drawable);
+            }
+            obj = (PyObject*)pyObj;
+            break;
+        }
+        default:
+            PyErr_SetString(PyExc_TypeError, "Unknown UIDrawable derived type");
+            return nullptr;
+    }
+    
+    if (type) {
+        Py_DECREF(type);
+    }
+    return obj;
+}
+
 int UICollectionIter::init(PyUICollectionIterObject* self, PyObject* args, PyObject* kwds)
 {
     PyErr_SetString(PyExc_TypeError, "UICollection cannot be instantiated: a C++ data source is required.");
@@ -16,6 +83,12 @@ int UICollectionIter::init(PyUICollectionIterObject* self, PyObject* args, PyObj
 
 PyObject* UICollectionIter::next(PyUICollectionIterObject* self)
 {
+    // Check if self and self->data are valid
+    if (!self || !self->data) {
+        PyErr_SetString(PyExc_RuntimeError, "Iterator object or data is null");
+        return NULL;
+    }
+    
     if (self->data->size() != self->start_size)
     {
         PyErr_SetString(PyExc_RuntimeError, "collection changed size during iteration");
@@ -35,9 +108,8 @@ PyObject* UICollectionIter::next(PyUICollectionIterObject* self)
         return NULL;
     }
     auto target = (*vec)[self->index-1];
-    // TODO build PyObject* of the correct UIDrawable subclass to return
+    // Return the proper Python object for this UIDrawable
-    //return py_instance(target);
+    return convertDrawableToPython(target);
-    return NULL;
 }
 
 PyObject* UICollectionIter::repr(PyUICollectionIterObject* self)
@@ -71,8 +143,7 @@ PyObject* UICollection::getitem(PyUICollectionObject* self, Py_ssize_t index) {
         return NULL;
     }
     auto target = (*vec)[index];
-    RET_PY_INSTANCE(target);
+    return convertDrawableToPython(target);
-return NULL;
 
 
 }
@@ -189,9 +260,18 @@ int UICollection::init(PyUICollectionObject* self, PyObject* args, PyObject* kwd
 
 PyObject* UICollection::iter(PyUICollectionObject* self)
 {
-    PyUICollectionIterObject* iterObj;
+    // Get the iterator type from the module to ensure we have the registered version
-    iterObj = (PyUICollectionIterObject*)PyUICollectionIterType.tp_alloc(&PyUICollectionIterType, 0);
+    PyTypeObject* iterType = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "UICollectionIter");
+    if (!iterType) {
+        PyErr_SetString(PyExc_RuntimeError, "Could not find UICollectionIter type in module");
+        return NULL;
+    }
+    
+    // Allocate new iterator instance
+    PyUICollectionIterObject* iterObj = (PyUICollectionIterObject*)iterType->tp_alloc(iterType, 0);
+    
     if (iterObj == NULL) {
+        Py_DECREF(iterType);
         return NULL;  // Failed to allocate memory for the iterator object
     }
 
@@ -199,5 +279,6 @@ PyObject* UICollection::iter(PyUICollectionObject* self)
     iterObj->index = 0;
     iterObj->start_size = self->data->size();
 
+    Py_DECREF(iterType);
     return (PyObject*)iterObj;
 }

src/UICollection.h

@@ -30,7 +30,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUICollectionIterType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.UICollectionIter",
         .tp_basicsize = sizeof(PyUICollectionIterObject),
         .tp_itemsize = 0,
@@ -44,9 +44,11 @@ namespace mcrfpydef {
         .tp_repr = (reprfunc)UICollectionIter::repr,
         .tp_flags = Py_TPFLAGS_DEFAULT,
         .tp_doc = PyDoc_STR("Iterator for a collection of UI objects"),
+        .tp_iter = PyObject_SelfIter,
         .tp_iternext = (iternextfunc)UICollectionIter::next,
         //.tp_getset = PyUICollection_getset,
         .tp_init = (initproc)UICollectionIter::init, // just raise an exception
+        .tp_alloc = PyType_GenericAlloc,
         //TODO - as static method, not inline lambda def, please
         .tp_new = [](PyTypeObject* type, PyObject* args, PyObject* kwds) -> PyObject*
         {
@@ -56,7 +58,7 @@ namespace mcrfpydef {
     };
 
 	static PyTypeObject PyUICollectionType = {
-		//PyVarObject_/HEAD_INIT(NULL, 0)
+		.ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
 		.tp_name = "mcrfpy.UICollection",
 		.tp_basicsize = sizeof(PyUICollectionObject),
 		.tp_itemsize = 0,

src/UIDrawable.h

@@ -57,59 +57,9 @@ typedef struct {
 } PyUICollectionIterObject;
 
 namespace mcrfpydef {
-    //PyObject* py_instance(std::shared_ptr<UIDrawable> source);
+    // DEPRECATED: RET_PY_INSTANCE macro has been replaced with template functions in PyObjectUtils.h
-    // This function segfaults on tp_alloc for an unknown reason, but works inline with mcrfpydef:: methods.
+    // The macro was difficult to debug and used static type references that could cause initialization order issues.
-
+    // Use PyObjectUtils::convertDrawableToPython() or PyObjectUtils::createPyObject<T>() instead.
-#define RET_PY_INSTANCE(target) {                       \
-switch (target->derived_type())                         \
-{                                                       \
-    case PyObjectsEnum::UIFRAME:                        \
-    {                                                   \
-        PyUIFrameObject* o = (PyUIFrameObject*)((&PyUIFrameType)->tp_alloc(&PyUIFrameType, 0)); \
-        if (o)                                          \
-        {                                               \
-            auto p = std::static_pointer_cast<UIFrame>(target); \
-            o->data = p;                                \
-            auto utarget = o->data;                     \
-        }                                               \
-        return (PyObject*)o;                            \
-    }                                                   \
-    case PyObjectsEnum::UICAPTION:                        \
-    {                                                   \
-        PyUICaptionObject* o = (PyUICaptionObject*)((&PyUICaptionType)->tp_alloc(&PyUICaptionType, 0)); \
-        if (o)                                          \
-        {                                               \
-            auto p = std::static_pointer_cast<UICaption>(target); \
-            o->data = p;                                \
-            auto utarget = o->data;                     \
-        }                                               \
-        return (PyObject*)o;                            \
-    }                                                   \
-    case PyObjectsEnum::UISPRITE:                        \
-    {                                                   \
-        PyUISpriteObject* o = (PyUISpriteObject*)((&PyUISpriteType)->tp_alloc(&PyUISpriteType, 0)); \
-        if (o)                                          \
-        {                                               \
-            auto p = std::static_pointer_cast<UISprite>(target); \
-            o->data = p;                                \
-            auto utarget = o->data;                     \
-        }                                               \
-        return (PyObject*)o;                            \
-    }                                                   \
-    case PyObjectsEnum::UIGRID:                        \
-    {                                                   \
-        PyUIGridObject* o = (PyUIGridObject*)((&PyUIGridType)->tp_alloc(&PyUIGridType, 0)); \
-        if (o)                                          \
-        {                                               \
-            auto p = std::static_pointer_cast<UIGrid>(target); \
-            o->data = p;                                \
-            auto utarget = o->data;                     \
-        }                                               \
-        return (PyObject*)o;                            \
-    }                                                   \
-}                                                       \
-}
-// end macro definition
 
 //TODO: add this method to class scope; move implementation to .cpp file
 /*

src/UIEntity.cpp

@@ -1,6 +1,7 @@
 #include "UIEntity.h"
 #include "UIGrid.h"
 #include "McRFPy_API.h"
+#include "PyObjectUtils.h"
 
 UIEntity::UIEntity() {} // this will not work lol. TODO remove default constructor by finding the shared pointer inits that use it
 
@@ -129,7 +130,9 @@ sf::Vector2i PyObject_to_sfVector2i(PyObject* obj) {
 
 // TODO - deprecate / remove this helper
 PyObject* UIGridPointState_to_PyObject(const UIGridPointState& state) {
-    return PyObject_New(PyObject, (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "GridPointState"));
+    // This function is incomplete - it creates an empty object without setting state data
+    // Should use PyObjectUtils::createGridPointState() instead
+    return PyObjectUtils::createPyObjectGeneric("GridPointState");
 }
 
 PyObject* UIGridPointStateVector_to_PyList(const std::vector<UIGridPointState>& vec) {

src/UIEntity.h

@@ -61,7 +61,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUIEntityType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Entity",
         .tp_basicsize = sizeof(PyUIEntityObject),
         .tp_itemsize = 0,

src/UIFrame.h

@@ -46,7 +46,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUIFrameType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Frame",
         .tp_basicsize = sizeof(PyUIFrameObject),
         .tp_itemsize = 0,

src/UIGrid.cpp

@@ -487,14 +487,17 @@ PyObject* UIEntityCollectionIter::next(PyUIEntityCollectionIterObject* self)
         PyErr_SetString(PyExc_RuntimeError, "the collection store returned a null pointer");
         return NULL;
     }
-    // Advance list iterator since Entities are not stored in a vector (if this code even worked)
+    // Advance list iterator since Entities are stored in a list, not a vector
-    // vectors only: //auto target = (*vec)[self->index-1];
+    auto l_begin = (*vec).begin();
-    //auto l_front = (*vec).begin();
+    std::advance(l_begin, self->index-1);
-    //std::advance(l_front, self->index-1);
+    auto target = *l_begin;
     
-    // TODO build PyObject* of the correct UIDrawable subclass to return
+    // Create and return a Python Entity object
-    //return py_instance(target);
+    auto type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "Entity");
-    return NULL;
+    auto o = (PyUIEntityObject*)type->tp_alloc(type, 0);
+    auto p = std::static_pointer_cast<UIEntity>(target);
+    o->data = p;
+    return (PyObject*)o;
 }
 
 PyObject* UIEntityCollectionIter::repr(PyUIEntityCollectionIterObject* self)
@@ -625,11 +628,18 @@ int UIEntityCollection::init(PyUIEntityCollectionObject* self, PyObject* args, P
 
 PyObject* UIEntityCollection::iter(PyUIEntityCollectionObject* self)
 {
-    //PyUIEntityCollectionIterObject* iterObj;
+    // Get the iterator type from the module to ensure we have the registered version
-    //iterObj = (PyUIEntityCollectionIterObject*)PyUIEntityCollectionIterType.tp_alloc(&PyUIEntityCollectionIterType, 0);
+    PyTypeObject* iterType = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "UIEntityCollectionIter");
-    auto type = (PyTypeObject*)PyObject_GetAttrString(McRFPy_API::mcrf_module, "EntityCollectionIter");
+    if (!iterType) {
-    auto iterObj = (PyUIEntityCollectionIterObject*)type->tp_alloc(type, 0);
+        PyErr_SetString(PyExc_RuntimeError, "Could not find UIEntityCollectionIter type in module");
+        return NULL;
+    }
+    
+    // Allocate new iterator instance
+    PyUIEntityCollectionIterObject* iterObj = (PyUIEntityCollectionIterObject*)iterType->tp_alloc(iterType, 0);
+    
     if (iterObj == NULL) {
+        Py_DECREF(iterType);
         return NULL;  // Failed to allocate memory for the iterator object
     }
 
@@ -637,5 +647,6 @@ PyObject* UIEntityCollection::iter(PyUIEntityCollectionObject* self)
     iterObj->index = 0;
     iterObj->start_size = self->data->size();
 
+    Py_DECREF(iterType);
     return (PyObject*)iterObj;
 }

src/UIGrid.h

@@ -99,7 +99,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUIGridType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Grid",
         .tp_basicsize = sizeof(PyUIGridObject),
         .tp_itemsize = 0,
@@ -130,8 +130,8 @@ namespace mcrfpydef {
     };
 
     static PyTypeObject PyUIEntityCollectionIterType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
-        .tp_name = "mcrfpy.UICollectionIter",
+        .tp_name = "mcrfpy.UIEntityCollectionIter",
         .tp_basicsize = sizeof(PyUIEntityCollectionIterObject),
         .tp_itemsize = 0,
         .tp_dealloc = (destructor)[](PyObject* self)
@@ -143,9 +143,11 @@ namespace mcrfpydef {
         .tp_repr = (reprfunc)UIEntityCollectionIter::repr,
         .tp_flags = Py_TPFLAGS_DEFAULT,
         .tp_doc = PyDoc_STR("Iterator for a collection of UI objects"),
+        .tp_iter = PyObject_SelfIter,
         .tp_iternext = (iternextfunc)UIEntityCollectionIter::next,
         //.tp_getset = UIEntityCollection::getset,
         .tp_init = (initproc)UIEntityCollectionIter::init, // just raise an exception
+        .tp_alloc = PyType_GenericAlloc,
         .tp_new = [](PyTypeObject* type, PyObject* args, PyObject* kwds) -> PyObject*
         {
             PyErr_SetString(PyExc_TypeError, "UICollection cannot be instantiated: a C++ data source is required.");
@@ -154,7 +156,7 @@ namespace mcrfpydef {
     };
 
     static PyTypeObject PyUIEntityCollectionType = {
-        //PyVarObject_/HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.EntityCollection",
         .tp_basicsize = sizeof(PyUIEntityCollectionObject),
         .tp_itemsize = 0,

src/UIGridPoint.h

@@ -66,7 +66,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUIGridPointType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.GridPoint",
         .tp_basicsize = sizeof(PyUIGridPointObject),
         .tp_itemsize = 0,
@@ -79,7 +79,7 @@ namespace mcrfpydef {
     };
 
     static PyTypeObject PyUIGridPointStateType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.GridPointState",
         .tp_basicsize = sizeof(PyUIGridPointStateObject),
         .tp_itemsize = 0,

src/UISprite.h

@@ -57,7 +57,7 @@ public:
 
 namespace mcrfpydef {
     static PyTypeObject PyUISpriteType = {
-        //PyVarObject_HEAD_INIT(NULL, 0)
+        .ob_base = {.ob_base = {.ob_refcnt = 1, .ob_type = NULL}, .ob_size = 0},
         .tp_name = "mcrfpy.Sprite",
         .tp_basicsize = sizeof(PyUISpriteObject),
         .tp_itemsize = 0,