Posted: 19th Apr 2021 1:33
Get3DVectorXFromScreen( X, Y ) doesn't appear to work with SetCameraFOV(1,0) (Orthographic Mode)

Suppose I could do it manually via OrthoWidth, AspectRatio, and CameraPosition...
Posted: 19th Apr 2021 2:07
You might want to have a look at these threads:

https://forum.thegamecreators.com/thread/221723
Posted: 19th Apr 2021 4:36
I found, after tinkering with it a bit, that it appears Get3DVectorXFromScreen() gives a position on a plane intersecting 0,0,0.
This is still useful, it saves me a few steps in getting a correct RayCast.
Here is the hack/fix I've used for my MouseRayCast() function:
+ Code Snippet
FUNCTION MouseRayCast(ObjID AS INTEGER, Depth AS INTEGER)
    MouseX AS FLOAT : MouseX = GetPointerX()
    MouseY AS FLOAT : MouseY = GetPointerY()

    Mouse3dX AS FLOAT : Mouse3dX = Get3DVectorXFromScreen( MouseX, MouseY )
    Mouse3dY AS FLOAT : Mouse3dY = Get3DVectorYFromScreen( MouseX, MouseY )
    Mouse3dZ AS FLOAT : Mouse3dZ = Get3DVectorZFromScreen( MouseX, MouseY )

    RayStartX AS FLOAT
    RayStartY AS FLOAT
    RayStartZ AS FLOAT
    RayEndX AS FLOAT
    RayEndY AS FLOAT
    RayEndZ AS FLOAT

    // When in Ortho Mode Get3DVectorXFromScreen() appears to give a position on a plane intersecting 0,0,0.  The plane is orientated as the Camera is.
    IF Viewport.IsOrtho[Viewport.Active]
        // Offset from Camera.Position
        RayStartX = GetCameraX(1) + Mouse3dX // @fix   offset to near clip plane
        RayStartY = GetCameraY(1) + Mouse3dY
        RayStartZ = GetCameraZ(1) + Mouse3dZ

        DeltaX AS FLOAT : DeltaX = Mouse3dX - RayStartX
        DeltaY AS FLOAT : DeltaY = Mouse3dY - RayStartY
        DeltaZ AS FLOAT : DeltaZ = Mouse3dZ - RayStartZ

        // Normalize
        DeltaL AS FLOAT : DeltaL = sqrt(DeltaX*DeltaX + DeltaY*DeltaY + DeltaZ*DeltaZ)
        DeltaX = DeltaX / DeltaL
        DeltaY = DeltaY / DeltaL
        DeltaZ = DeltaZ / DeltaL

        RayEndX = RayStartX + ( DeltaX * Depth )
        RayEndY = RayStartY + ( DeltaY * Depth )
        RayEndZ = RayStartZ + ( DeltaZ * Depth )
    ELSE
        RayStartX = GetCameraX(1) // @fix  offset to near clip plane
        RayStartY = GetCameraY(1)
        RayStartZ = GetCameraZ(1)

        RayEndX = RayStartX + ( Mouse3dX * Depth )
        RayEndY = RayStartY + ( Mouse3dY * Depth )
        RayEndZ = RayStartZ + ( Mouse3dZ * Depth )
    ENDIF

    TheObjectHit AS INTEGER : TheObjectHit = ObjectRayCast( ObjID, RayStartX, RayStartY, RayStartZ, RayEndX, RayEndY, RayEndZ )
ENDFUNCTION TheObjectHit
Posted: 19th Apr 2021 5:31
Here is a version that supports near clipping:

+ Code Snippet
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
TYPE MouseType
    PosX     AS FLOAT
    PosY     AS FLOAT

    PosOldX  AS FLOAT
    PosOldY  AS FLOAT

    PosLockX AS FLOAT
    PosLockY AS FLOAT

    DeltaX   AS FLOAT
    DeltaY   AS FLOAT

    LockPos  AS INTEGER
ENDTYPE
GLOBAL Mouse AS MouseType


//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
FUNCTION MouseRayCast(AGK_ObjectID AS INTEGER, DepthStart AS INTEGER, DepthEnd AS INTEGER)
    MouseCastX AS FLOAT : MouseCastX = Get3DVectorXFromScreen( Mouse.PosX, Mouse.PosY )
    MouseCastY AS FLOAT : MouseCastY = Get3DVectorYFromScreen( Mouse.PosX, Mouse.PosY )
    MouseCastZ AS FLOAT : MouseCastZ = Get3DVectorZFromScreen( Mouse.PosX, Mouse.PosY )

    RayStartX AS FLOAT
    RayStartY AS FLOAT
    RayStartZ AS FLOAT

    IF Viewport.IsOrtho[Viewport.Active]
        // When in Ortho Mode Get3DVectorXFromScreen() appears to give a position on a plane intersecting 0,0,0.  This plane is orientated as the Camera is.

        // Offset from Viewport.Pos
        RayStartX =  Viewport.Pos[Viewport.Active].x + MouseCastX
        RayStartY =  Viewport.Pos[Viewport.Active].y + MouseCastY
        RayStartZ = -Viewport.Pos[Viewport.Active].z + MouseCastZ  // Z is inverted in AGK :(

        // Get Delta
        MouseCastX = MouseCastX - RayStartX
        MouseCastY = MouseCastY - RayStartY
        MouseCastZ = MouseCastZ - RayStartZ

        // Normalize
        DeltaL AS FLOAT : DeltaL = sqrt(MouseCastX*MouseCastX + MouseCastY*MouseCastY + MouseCastZ*MouseCastZ)
        MouseCastX = MouseCastX / DeltaL
        MouseCastY = MouseCastY / DeltaL
        MouseCastZ = MouseCastZ / DeltaL

        RayStartX = RayStartX + (MouseCastX * DepthStart)
        RayStartY = RayStartY + (MouseCastY * DepthStart)
        RayStartZ = RayStartZ + (MouseCastZ * DepthStart)
    ELSE
        RayStartX =  Viewport.Pos[Viewport.Active].x + (MouseCastX * DepthStart)
        RayStartY =  Viewport.Pos[Viewport.Active].y + (MouseCastY * DepthStart)
        RayStartZ = -Viewport.Pos[Viewport.Active].z + (MouseCastZ * DepthStart) // Z is inverted in AGK :(
    ENDIF

    RayEndX AS FLOAT : RayEndX = RayStartX + ( MouseCastX * DepthEnd )
    RayEndY AS FLOAT : RayEndY = RayStartY + ( MouseCastY * DepthEnd )
    RayEndZ AS FLOAT : RayEndZ = RayStartZ + ( MouseCastZ * DepthEnd )

    AGK_ObjectHitID AS INTEGER : AGK_ObjectHitID = ObjectRayCast( AGK_ObjectID, RayStartX, RayStartY, RayStartZ, RayEndX, RayEndY, RayEndZ )
ENDFUNCTION AGK_ObjectHitID


//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
FUNCTION SetMouseLock(SetTo AS INTEGER)
    IF SetTo
        Mouse.LockPos = 1
        Mouse.PosLockX = GetRawMouseX()
        Mouse.PosLockY = GetRawMouseY()
    ELSE
        Mouse.LockPos = 0
        Mouse.PosLockX = 0.0
        Mouse.PosLockY = 0.0
    ENDIF
ENDFUNCTION


//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
FUNCTION UpdateInputMouse()
    IF Mouse.LockPos
        Mouse.PosX = GetRawMouseX()
        Mouse.PosY = GetRawMouseY()

        SetRawMousePosition( Mouse.PosLockX, Mouse.PosLockY )

        Mouse.PosOldX = GetRawMouseX()
        Mouse.PosOldY = GetRawMouseY()
    ELSE
        Mouse.PosOldX = Mouse.PosX
        Mouse.PosOldY = Mouse.PosY

        Mouse.PosX = GetRawMouseX()
        Mouse.PosY = GetRawMouseY()
    ENDIF
    Mouse.DeltaX = Mouse.PosX - Mouse.PosOldX
    Mouse.DeltaY = Mouse.PosY - Mouse.PosOldY
ENDFUNCTION

It uses some of my internal code, but I'm sure you can figure it out if you need it.