Berkshire Hathaway will not exercise its $5 billion of warrants in Goldman Sachs immediately, even though doing so would result in a profit of nearly $2 billion, billionaire investor Warren Buffett told CNBC.

The news, which comes the day after Goldman notified Berkshire that it will call its $5 billion in Goldman preferred shares in 30 days, signals that the Oracle of Omaha thinks Goldman stock is worth even more than the roughly $160 a share it trades at today. Berkshire's warrants come with an exercise price $115 a share.

"I think the odds are that it goes higher," Buffett, the chairman and CEO of Berkshire Hathaway, said. "Over time most stocks will get more valuable, and I have no reason to think Goldman Sachs is an exception to that."

Of course, with this investment the clock is ticking. Buffett's Goldman warrants expire in 2013.

Buffett made the original investment in Goldman at the height of the financial crisis in the fall of 2008. He had passed on many deals that came his way during that time, and so his decision to invest in Goldman helped shore up market sentiment in the investment bank at the time.

But it came at a steep price for the bank: Berkshire's shares come with a dividend of 10 percent, roughly $500 million a year.

Goldman was eager to end the arrangement. It acted to buy back the shares the same day that it — and other banks — received approval from the Federal Reserve to buy back shares and raise their dividends.

Buffett also made a big bet in General Electric during the same time period, buying $3 billion in GE preferred shares that also came with a 10 percent dividend. General Electric cannot redeem those shares until this fall.

"GE has said they're going to call the preferred as soon a they can, which I believe is some day in October this year," Buffett added.

GE is a minority shareholder in NBC Universal, CNBC's parent company.

Current Berkshire stock prices:

Class B:

Class A:

Email comments to buffettwatch@cnbc.com