cannot access server's port (localhost from same router, direct ip access from other router)

[Pleizz]

i wanna open port by using nginx only if packet from my server's ip.
so, i've coded like

allow [my ip];
deny all;

yeah, it works if i connect with wifi (same router with vpn server).
but didn't work if i connect with openvpn.

can i get some advices?

p.s. i've tried convert udp <==> tcp, dns

[Pleizz]

direct access to private ip works fine but not public ip & domain. T.T