I'm not sure what that thing under your sink is.
If you have a water softener somewhere in the house there is usually a garden hose connection right before it actually goes into the softener, hooking it up there is usually easier ime.
Otherwise yeah, just use a T or Y style splitter, just make sure the water coming out of the faucet IS NOT RODI. Some builds will have you pump the rodi water out the faucet like it's an RO water filter for household use, will likely make you very sick long term.
If its RO water it wont hurt you and is good even for babies.
If its RODI water, there is some evidence the water will interact with open sores in the GI tract or strip nutrients from your body. I would never let my kids drink RODI