Nvidia shares fell about 7% in premarket trading on Thursday, according to CNBC, after the company's board met on Wednesday to discuss whether to remove Jensen Huang as CEO. CNBC confirmed the board meeting.

Huang admitted to an affair on Sunday, according to the source. The Financial Times reported that he was romantically involved with Alice Huang, an executive at cloud company Megaspeed. U.S. intelligence officials believe Alice Huang is a spy for China, according to the Financial Times. The source does not say whether Nvidia or Huang has responded to that belief.

The New York Times reported on Wednesday that Huang had another extramarital relationship, with Kimberly Powell, Nvidia's vice president of health care. The Times reported that Powell reports to Huang.

Several Wall Street analysts told CNBC that if Huang is removed, the stock could fall 20%. Nvidia's stock is already down more than 20% over the last month. Bernstein cut its price target on Nvidia from $400 to $200.

Stacy Rasgon, a senior analyst at Bernstein, told CNBC's "Squawk Box Asia" that the alleged affair with a direct report makes it "much tougher for the board to leave him in position." He also said it is unclear who could lead Nvidia as a new CEO.

U.S. lawmakers have called for Huang's resignation, according to the source. Richard Reyle, chief investment officer at Questar Capital Partners, told CNBC that Nvidia's lack of a succession plan makes the stock "uninvestable."