This way the water dispenser should send water to the ice maker. If it does it means that the dispenser lever switch is working and the water valve for the dispenser is working.
If you do get water to the ice maker doing this your water line could be frozen in the door. You can disconnect the line under the unit before it goes into the door at the bottom hinge. Push the dispenser button and see if it comes out there. If it does try a hair dryer blowing where the water line is at the dispenser for a few minutes. Raising the temp of freezer slightly might prevent it from happening again.