Roger Martinez and Duvan Zapata scored in the final 20 minutes after an Argentina side featuring Lionel Messi and Sergio Aguero had dominated.
Messi, 31, said the Group B defeat left his side “feeling bitter”.
“We didn’t want to start this way,” the five-time Ballon d’Or winner said.
“But now we have to lift our heads and keep going.”
Argentina has won the tournament 14 times – second only to Uruguay – but have not triumphed since 1993 and lost the last two finals on penalties against Chile.
Barcelona’s Messi missed his side’s best chance to score in Salvador, heading a rebound wide after David Ospina had saved from Manchester City defender Nicolas Otamendi.
Colombia, managed by former Portugal and Real Madrid head coach Carlos Quieroz, scored both of their goals against the run of play.
Substitute Martinez cut in from the left and drove his effort into the top corner in the 71st minute after being set up by James Rodriguez.
Zapata, who scored 23 goals to help his Italian side Atalanta qualify for the Champions League for the first time last season, sealed the win with a late close-range effort, having replaced captain Radamel Falcao.
Argentina next face Paraguay, who began their campaign against invitees Qatar last night.
In Saturday’s earlier game, Venezuela and Peru playe
Peru had two goals ruled out by VAR while Luis Mago was sent off for Venezuela for a second yellow card late on.
Hosts Brazil top Group A following their 3-0 win over Bolivia.