The research on the method of the tennis ball tracking and judging is enlightened by the “Hawk eye system” in the tennis match. When the athletes have different opinions on the judging of the ball, they can use the “Hawk eye system” to challenge the umpire. Then the trajectory of ball is presented on the screen, after that the final judgment of that point is made. But due to the excessive price of that system and time consuming procedures when installing, it is only possible to use that system in the significant matches. So this thesis is concentration on how to simplify that system by using the normal camera and lap-top computer to give solution to the tennis ball tracking and judging issues. The problem to solve first is that to recognize the tennis ball correctly in the video sequence, at the same time this thesis is going to compare several methods of recognizing the tennis ball in motion, after that erasing the background noise and tracking the tennis trajectory. After already obtaining the trajectory of the tennis ball, the mathematical method is used to get the inflection point. Later we recognize the base line in the tennis court by comparing several different techniques, then computing whether the inflection point is in the base line or out the base line even on the base line. When getting the result, the judgment is made according to the rules of the tennis match. In the end of the thesis, some experiment results are given to demonstrate that the system works well overall and analyze the results of the experiment of the method. Finally some suggestion of improving the system and opinions on the future work are presented.