Most Agile methodologies provide for the application of a Technical Spike for exploring a new technology or risky areas of a product. The Scaled Agile Framework (SAFe) methodology defines the Spike as a type of Exploration and Enabler Story.
There are a number of approaches that have been recommended for Technical Spikes. These include:
Estimating and sizing a Technical Spike Story
Timeboxing a Technical Spike .
The Technical Spike is exploratory in nature and it is conflicting to attempt to estimate the complexity for a piece of work that is not well understood; in my experience most teams would rather Timebox the efforts required to complete a Technical Spike.