To prove that the necessary and sufficient condition for a non-empty subset $H$ of a group $(G, \ast)$ to be a subgroup is:

$a, b \in H \Rightarrow a \ast b^{-1} \in H,$

we need to verify both directions of the implication: necessity and sufficiency.

Sufficiency

Assume that $H$ is a non-empty subset of $G$ and satisfies the condition: $a, b \in H \Rightarrow a \ast b^{-1} \in H.$ To show that $H$ is a subgroup of $G$, we need to demonstrate that $H$ satisfies the subgroup criteria:

1. Closure under the group operation: If $a, b \in H$, then $a \ast b \in H$.
2. Existence of the identity element in $H$: There exists an element $e \in H$ such that for any $a \in H$, $e \ast a = a \ast e = a$.
3. Existence of inverses for each element in $H$: For each $a \in H$, there exists an $a^{-1} \in H$ such that $a \ast a^{-1} = e$.

Since $H$ is non-empty, let $a \in H$. Setting $a = b$ in the condition $a, b \in H \Rightarrow a \ast b^{-1} \in H$, we get: $a \ast a^{-1} \in H.$ Since $a \ast a^{-1} = e$, where $e$ is the identity element of $G$, we conclude that $e \in H$.

Now, to show closure, let $a, b \in H$. Then, using the given condition, we have: $a \ast b^{-1} \in H.$ Since $e \in H$, taking $a = e$ in the condition implies that for any $b \in H$, $b^{-1} \in H$. Therefore, $H$ contains the inverse of each of its elements.

Finally, for closure under the operation, consider any $a, b \in H$. We have shown that $b^{-1} \in H$, so by applying the condition again, $a \ast b = a \ast (b^{-1})^{-1} \in H$.

Thus, $H$ is a subgroup of $G$ since it satisfies the subgroup criteria.

Necessity

Now, we prove that if $H$ is a subgroup of $G$, then it satisfies the condition $a, b \in H \Rightarrow a \ast b^{-1} \in H$.

If $H$ is a subgroup, then by definition:

1. $H$ contains the identity element $e$.
2. $H$ is closed under the group operation $\ast$.
3. For any $a \in H$, $a^{-1} \in H$.

Now, if $a, b \in H$, then $b^{-1} \in H$ because $H$ is closed under inverses. Since $H$ is also closed under the group operation, $a \ast b^{-1} \in H$.

Thus, the condition $a, b \in H \Rightarrow a \ast b^{-1} \in H$ holds.

Conclusion

We have shown that the condition $a, b \in H \Rightarrow a \ast b^{-1} \in H$ is both necessary and sufficient for $H$ to be a subgroup of $G$.

Would you like further details on any part of this proof?

Here are some related questions for further exploration:

1. How does this criterion compare to the three-step subgroup test for subgroups?
2. Can this criterion be generalized to non-abelian groups as well?
3. Why does this criterion require the set $H$ to be non-empty?
4. What changes, if any, are needed for this proof if $G$ is not finite?
5. How does this criterion relate to the definition of a normal subgroup?

Tip: Remember, testing if a subset $H$ satisfies $a, b \in H \Rightarrow a \ast b^{-1} \in H$ can sometimes simplify subgroup verification compared to checking each subgroup property individually.