UnProject extension method in Viewport3DX

I am trying to use the UnProject extension method available on the Viewport3DX class in order to convert a 2D mouse coordinate to a 3D Point/Ray. I find that I'm able to accurately compute the ray/hit point only for mouse clicks on the X-axis but if I click on anywhere else on the grid the computed ray and hit point in 3D coordinates are off. Any idea on what could be missing here?