'Under weigh' is unusual, but acceptable, spelling of the term.
I find that quite odd, since a vessel is said to "have way" on her when she is in motion (in any direction, desired or not), and "give way" is to push off from the dock, i.e., to first gain motion. She can get "under way" once she "weighs anchor" -- I imagine the sense of that deriving from the fact that when the weight of the anchor can be felt by the crew raising it, it has left the bottom and is no longer, um, anchoring the ship. So maybe "under weigh" is found acceptable due to a conflation of homonyms, and the sense may be "under [the influence of having] weigh[ed anchor]".
People often write, "We'll try a different tact" instead of "tack", I think by a confusion of tack, the nautical term, with tactic, since the metaphor lies in using "physical approach'" (tack) as a stand-in for "strategies" (tactics).
It's kind of interesting, watching language evolve.