getIMFData.m

function [output] = getIMFData(database_id, series_id, countrycode2L, frequency, observation_start, observation_end, vintage_id, adinput1, adinput2, adinput3,adinput4)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% database_id: currently supports 'BOP', 'IFS', 'DOT', 'CPIS', 'FSI', 'CPI'
%   and all seven GFS: 'GFSMAB', 'GFSE', 'GFSCOFOG', 'GFSR', 'GFSSSUC','GFSFALCS','GFSIBS'
% [IMF datasets Balance of Payments (BOP), International Fiscal Statistics (IFS), 
% Direction of Trade Statistics (DOT), Coordinated Portfolio Investment Survey (CPIS),
% Financial Soundness Indicators (FSI), Consumer Price Index (CPI), and the seven
% Government Finance Statistics (GFS): GFS Main Aggregates and Balances (GFSMAB), 
% GFS Expense (GFSE), GFS Classification of the Functions of Government (GFSCOFOG), 
% GFS Revenue (GFSR), GFS Statement of Sources and Uses of Cash (GFSSSUC),
% GFS Financial Assets and Liabilities by Counterpart Sector (GFSFALCS), 
% GFS Integrated Balance Sheet (GFSIBS).]
%
% If the only input is the database_id, then output will return a
% dictionary of 'series_id' (and other inputs) and their names. 
% This can then be searched to find the series_id for a variable when 
% you only have some idea what the name might be.
%
% -----------
% Inputs:
%   series_id: the code for the variable you want
%   frequency: for example monthly M, quarterly Q, or annually A;
%
%   If you do not input observation_start and observation_end you will be
%   given data for all available dates. Equivalently, set
%   observation_start=[], and similarly for observation_end=[].
%
%   For BOP, FSI, IFS, CPI databases no further inputs are required. 
%   Otherwise the additional inputs depend on the database.
%   The additional inputs come after vintage_id, which can typically just be left empty, i.e., vintage_id=[]
%   DOT uses one further input
%   CPIS uses three further inputs. 
%   GFSMAB,GFSE,GFSCOFOG,GFSR,GFSSSUC use two further inputs.
%   GFSFALCS uses three further inputs.
%   GFSIBS uses four further inputs.
% -----------
% The further inputs are:
%   adinput1, adinput2, adinput3, adinput4
%   They are interpreted in this order depending on the database.
%   If not used, just don't include these inputs.
% Using DOT you will need one further input (adinput1)
%   counterpartycountrycode2L: same format as countrycode2L, is the counterpart country
% Using CPIS you will need three further inputs (adinput1, adinput2, adinput3):
%   sector_id: see CPIS database for an explanation of Sectors
%   counterpartycountrycode2L: same format as countrycode2L, is the counterpart country
%   counterparty_id: same format as sector_id, is the counterpart sector
% Using GFSMAB,GFSE,GFSCOFOG,GFSR,GFSSSUC you need two further inputs (adinput1, adinput2):
%   sector_id: see GFS database for an explanation of Sectors
%   unit: see GFS database for an explanation of units
% Using GFSFALCS you need three further inputs (adinput1, adinput2, adinput3):
%   sector_id: see GFS database for an explanation of Sectors
%   unit: see GFS database for an explanation of units
%   counterparty_id: see GFS database for an exlanation of counterparty, is the counterpart sector (Central Bank, Banks ...)
% Using GFSIBS you need three further inputs (adinput1, adinput2, adinput3):
%   sector_id: see GFS database for an explanation of Sectors
%   unit: see GFS database for an explanation of units
%   instr_id:  see GFS database for an explanation of instr_id
%   counterparty_id: see GFS database for an explanation of counterparty_id (total, domestic / external)
% Note: counterparty_id means different things for the different databases.
%
%
% To access vintage data you need to include a 'vintage_id'. This will be a
% string that contains the year and, where relevant, either quarter or month for the vintage
% you want, e.g., '2017', '2017Q1' or '2017M07'. Note, some databases are just
% annual, some quarterly, and some monthly. 
% 
% I have deliberately made it so that the actual output is similar to that
% from getFredData(). Especially that the dates and data are in output.Data
%
% Some examples of usage can be found on my website, or on my github.
%
% 2019
% Robert Kirkby
% robertdkirkby.com
% Special thanks to Mikhail Nicolaev for implementing the GFS databases.
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%     

%%%%%
%
% Format of the request sent to the api:
%   BOP: e.g., 'NZ. ' New Zealand 
%   IFS: e.g., '
%   DOT: e.g., 'IT. .FR'; Italy-France, 
%
% Connects to IMF database and retrieves the data series identified by series_id. 
% See https://www.bd-econ.com/imfapi1.html
% 
% The IMF's CompactData method, combined with codes for the series, frequency, area, and indicator, returns a JSON structured dataset. The codes and method are explained in more detail as follows:
%
% Area: The country, region, or set of countries, for example GB for the U.K., or GB+US for the U.K. and the U.S.;
% Indicator: The code for the indicator of interest--IFS includes more than 2,500. In the example above, the code of interest is PMP_IX; and
% Date Range (Optional): Use this to limit the data range returned, for example ?startPeriod=2010&endPeriod=2017 otherwise the full set of data is returned.
%
% http://dataservices.imf.org/REST/SDMX_JSON.svc/CompactData/DS-BOP/A.NZ.?startPeriod={start date}&endPeriod={end date}
% http://dataservices.imf.org/REST/SDMX_JSON.svc/CodeList/{dimension_list[2]}
%
% Method: CompactData retrieves data, DataStructure retrieves series information, and GenericMetadata returns the metadata;
%
% Three websites I found useful to figure out the 'codes' for making a request
% http://datahelp.imf.org/knowledgebase/articles/630877-api
% http://datahelp.imf.org/knowledgebase/articles/667681-using-json-restful-web-service
% http://dataservices.imf.org/REST/SDMX_JSON.svc/help
%
% For Balance of Payments data, the database ID is 'BOP' (database 81, see full list of databases at
% http://dataservices.imf.org/REST/SDMX_JSON.svc/Dataflow 
% Use regular IMF website to find the name of the database, and then use above link to find the relevant database ID code.
%
%%%%%


%%
database_id_full=database_id;
if exist('vintage_id', 'var')==1
   if ~isempty(vintage_id)
       database_id_full=[database_id,'_',vintage_id];
       % Awkwardly some datasets use 'M07' for July while others use 'M7', the
       % following lines deal with this for the user who always just puts in
       % 'M07'
       try str2num(vintage_id(end-1)) % If the second last is a number
           if str2num(vintage_id(end-1))==0 % And that number is a zero
               try webread(['http://dataservices.imf.org/REST/SDMX_JSON.svc/DataStructure/',database_id_full])
                   % If it works, do nothing
               catch % If it does not work, then remove the zero.
                   vintage_id=[vintage_id(1:end-2),vintage_id(end)];
               end
           end
       catch
           % If it second last is not a number then there is nothing to worry about (hopefully!!!)
       end
   end
end

%%
if nargin==1 % Just return a dictionary for that database
    % /DataStructure/
    JSONdata2 = webread(['http://dataservices.imf.org/REST/SDMX_JSON.svc/DataStructure/',database_id_full]);
    
    temp=JSONdata2.Structure.CodeLists.CodeList{1}.Code;
    for ii=1:length(temp)
        output.Scale{ii,1}=temp(ii).x_value;
        output.Scale{ii,2}=temp(ii).Description.x_text;
    end
    temp=JSONdata2.Structure.CodeLists.CodeList{2}.Code;
    for ii=1:length(temp)
        output.Frequency{ii,1}=temp(ii).x_value;
        output.Frequency{ii,2}=temp(ii).Description.x_text;
    end
    temp=JSONdata2.Structure.CodeLists.CodeList{3}.Code;
    for ii=1:length(temp)
        output.CountryCodes{ii,1}=temp(ii).x_value;
        output.CountryCodes{ii,2}=temp(ii).Description.x_text;
    end
    % The above first three appear to always be the same, at least for BOP, IFS, CPIS and FSI. 
    % After the first three things are specific to the database being queried.
    if strcmp(database_id,'IFS') || strcmp(database_id,'FSI')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{5}.Code;
        for ii=1:length(temp)
            output.TimeFormat{ii,1}=temp(ii).x_value;
            output.TimeFormat{ii,2}=temp(ii).Description.x_text;
        end
    end
    if strcmp(database_id,'CPIS')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{5}.Code;
        for ii=1:length(temp)
            output.Sector{ii,1}=temp(ii).x_value;
            output.Sector{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{6}.Code;
        for ii=1:length(temp)
            output.TimeFormat{ii,1}=temp(ii).x_value;
            output.TimeFormat{ii,2}=temp(ii).Description.x_text;
        end
    end
    if strcmp(database_id,'DOT')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{5}.Code;
        for ii=1:length(temp)
            output.CouterpartCountry{ii,1}=temp(ii).x_value;
            output.CouterpartCountry{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{6}.Code;
        for ii=1:length(temp)
            output.TimeFormat{ii,1}=temp(ii).x_value;
            output.TimeFormat{ii,2}=temp(ii).Description.x_text;
        end
    end
    if strcmp(database_id,'CPI')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
    end
    if strcmp(database_id,'GFSMAB') || strcmp(database_id,'GFSE') || strcmp(database_id,'GFSCOFOG') || strcmp(database_id,'GFSR') || strcmp(database_id,'GFSSSUC')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Sector{ii,1}=temp(ii).x_value;
            output.Sector{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{5}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{6}.Code;
        for ii=1:length(temp)
            output.Unit{ii,1}=temp(ii).x_value;
            output.Unit{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{7}.Code;
        for ii=1:length(temp)
            output.TimeFormat{ii,1}=temp(ii).x_value;
            output.TimeFormat{ii,2}=temp(ii).Description.x_text;
        end
    end
    if strcmp(database_id,'GFSFALCS')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Sector{ii,1}=temp(ii).x_value;
            output.Sector{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{5}.Code;
        for ii=1:length(temp)
            output.Unit{ii,1}=temp(ii).x_value;
            output.Unit{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{6}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{7}.Code;
        for ii=1:length(temp)
            output.Couterpart{ii,1}=temp(ii).x_value;
            output.Couterpart{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{8}.Code;
        for ii=1:length(temp)
            output.TimeFormat{ii,1}=temp(ii).x_value;
            output.TimeFormat{ii,2}=temp(ii).Description.x_text;
        end
    end
    if strcmp(database_id,'GFSIBS')
        temp=JSONdata2.Structure.CodeLists.CodeList{4}.Code;
        for ii=1:length(temp)
            output.Sector{ii,1}=temp(ii).x_value;
            output.Sector{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{5}.Code;
        for ii=1:length(temp)
            output.Unit{ii,1}=temp(ii).x_value;
            output.Unit{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{6}.Code;
        for ii=1:length(temp)
            output.Variables{ii,1}=temp(ii).x_value;
            output.Variables{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{7}.Code;
        for ii=1:length(temp)
            output.Instrument{ii,1}=temp(ii).x_value;
            output.Instrument{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{8}.Code;
        for ii=1:length(temp)
            output.Couterpart{ii,1}=temp(ii).x_value;
            output.Couterpart{ii,2}=temp(ii).Description.x_text;
        end
        temp=JSONdata2.Structure.CodeLists.CodeList{9}.Code;
        for ii=1:length(temp)
            output.TimeFormat{ii,1}=temp(ii).x_value;
            output.TimeFormat{ii,2}=temp(ii).Description.x_text;
        end
    end
    return
end


%% Figure out the additional inputs based on which database is being used.
if strcmp(database_id,'IFS') || strcmp(database_id,'FSI') || strcmp(database_id,'BOP') || strcmp(database_id,'CPI')
    % No additional inputs
    if nargin>7
        error('Too many inputs given the database you are using')
    end
elseif strcmp(database_id,'DOT')
    counterpartycountrycode2L=adinput1;
    if nargin>8
        error('Too many inputs given the database you are using')
    end
elseif strcmp(database_id,'CPIS')
    counterpartycountrycode2L=adinput1;
    sector_id=adinput2;
    counterparty_id=adinput3; % counterparty-sector
    if nargin>10
        error('Too many inputs given the database you are using')
    end
    % Because I introduced a breaking change (June 2022) to CPIS, reordering inputs.
    % Check if vintageid looks like it is supposed to be counterpartycountrycode2L and throw a warning
    if iscell(vintage_id)
        if all(isletter(vintage_id{1}))
            warning('CPIS input ordering was changed in June 2022, you appear to be using the old ordering. See CPISexample code on project github for how to order inputs')
            % Github link: https://github.com/robertdkirkby/getimfdata-matlab/blob/master/getIMFData_CPISexample.m
        end
    elseif isempty(vintage_id)
        % Do nothing, this is likely intended (had to seperate this case as empty is considered a letter by matlab).
    elseif all(isletter(vintage_id))
        warning('CPIS input ordering was changed in June 2022, you appear to be using the old ordering. See CPISexample code on project github for how to order inputs')
        % Github link: https://github.com/robertdkirkby/getimfdata-matlab/blob/master/getIMFData_CPISexample.m
    end
elseif strcmp(database_id,'GFSMAB') || strcmp(database_id,'GFSE') || strcmp(database_id,'GFSCOFOG') || strcmp(database_id,'GFSR') || strcmp(database_id,'GFSSSUC')
    sector_id=adinput1;
    unit=adinput2;
    if nargin>9
        error('Too many inputs given the database you are using')
    end
elseif strcmp(database_id,'GFSFALCS')
    sector_id=adinput1;
    unit=adinput2;
    counterparty_id=adinput3; % Central Bank, Banks ...
    if nargin>10
        error('Too many inputs given the database you are using')
    end
elseif strcmp(database_id, 'GFSIBS')
    sector_id=adinput1;
    unit=adinput2;
    instr_id=adinput3;
    counterparty_id=adinput4;  % total, domestic / external
    % Note: counterpartysector_id means different things for CPIS, GFSFALCS and GFSIBS
end

%% Not just database_id, so an actual data 'series_id' has been requested
if strcmp(database_id,'IFS') || strcmp(database_id, 'BOP') || strcmp(database_id, 'FSI') || strcmp(database_id, 'CPI')
    optionstring=[frequency,'.',countrycode2L,'.',series_id];
    if nargin==7
        optionstring=[frequency,'.',countrycode2L,'.',series_id,'.',timeformat];
    end
elseif strcmp(database_id,'CPIS')
    optionstring=[frequency,'.',countrycode2L,'.',series_id,'.',sector_id,'.',counterparty_id,'.',counterpartycountrycode2L];
elseif strcmp(database_id,'DOT')
    optionstring=[frequency,'.',countrycode2L,'.',series_id,'.',counterpartycountrycode2L];
elseif strcmp(database_id, 'GFSMAB') || strcmp(database_id, 'GFSCOFOG') || strcmp(database_id, 'GFSE') || strcmp(database_id, 'GFSR') || strcmp(database_id, 'GFSSSUC')
    optionstring=[frequency,'.',countrycode2L,'.',sector_id,'.',unit,'.',series_id];
elseif strcmp(database_id, 'GFSFALCS')
    optionstring=[frequency,'.',countrycode2L,'.',sector_id,'.',unit,'.',series_id,'.',counterparty_id]; %counterpart_sector - Central Bank, Banks ...
elseif strcmp(database_id, 'GFSIBS')
    optionstring=[frequency,'.',countrycode2L,'.',sector_id,'.',unit,'.',series_id,'.',instr_id,'.',counterparty_id]; %counterpart_sector2 - total, domestic / external
end

if exist('observation_start','var')==1 && exist('observation_end','var')==1 % if specific start and end dates are given
    if ~isempty(observation_start)
        longoptionstring=[optionstring,'?startPeriod=',observation_start,'&endPeriod=',observation_end];
    end
else
    longoptionstring=optionstring;
end

%% /CompactData/
% For some reason the JSON sometimes is not read correctly, and is interpreted as text. When this occurs I rerequest the JSON.
JSONdata = webread(['http://dataservices.imf.org/REST/SDMX_JSON.svc/CompactData/',database_id_full,'/',longoptionstring]);
attempts=0;
while ~isstruct(JSONdata) && attempts<=10 % Try a few time to get the data as JSON (rather than text, which for some reason sometimes happens)
    JSONdata = webread(['http://dataservices.imf.org/REST/SDMX_JSON.svc/CompactData/',database_id_full,'/',longoptionstring]);
    attempts=attempts+1;
    fprintf('getIMFData: the request to IMF failed to parse, reattempting (this msg is purely FYI, not indicative of eventual error) \n')
    if attempts==10 && ~isstruct(JSONdata)
        fprintf('ERROR, getIMFData: The request to IMF keeps failing to parse, typically this is solved by just rerunning (there is no discernable reason for this error)')
    end
end
% Sort the data into output
if isfield(JSONdata.CompactData.DataSet,'Series')
    output.Data=nan(length(JSONdata.CompactData.DataSet.Series.Obs),2);
    temp=JSONdata.CompactData.DataSet.Series.Obs;
    % for unknown reason 'temp' is sometimes a cell and sometimes a structure
    if iscell(temp)
        for ii=1:length(JSONdata.CompactData.DataSet.Series.Obs)
            if isfield(JSONdata.CompactData.DataSet.Series.Obs{ii},'x_OBS_VALUE')
                output.Data(ii,2)=str2num(JSONdata.CompactData.DataSet.Series.Obs{ii}.x_OBS_VALUE);
            else
                % Just leave it as nan
                fprintf('getIMFDATA Warning: One of the time periods you requested is missing data, have returned a nan value. \n');
            end
            tempstr{ii}=JSONdata.CompactData.DataSet.Series.Obs{ii}.x_TIME_PERIOD; % I have deliberately chosen a format for the .Data that is like the output of getFREDdata()
        end
    else % otherwise it is a structure
        for ii=1:length(JSONdata.CompactData.DataSet.Series.Obs)
            if isfield(JSONdata.CompactData.DataSet.Series.Obs(ii),'x_OBS_VALUE')
                output.Data(ii,2)=str2num(JSONdata.CompactData.DataSet.Series.Obs(ii).x_OBS_VALUE);
            else
                % Just leave it as nan
                fprintf('getIMFDATA Warning: One of the time periods you requested is missing data, have returned a nan value. \n');
            end
            tempstr{ii}=JSONdata.CompactData.DataSet.Series.Obs(ii).x_TIME_PERIOD; % I have deliberately chosen a format for the .Data that is like the output of getFREDdata()
        end
    end
    for ii=1:length(JSONdata.CompactData.DataSet.Series.Obs)
        if strcmp(frequency,'A')
            output.Data(ii,1)=datenum(str2num(tempstr{ii}),1,1);
        elseif strcmp(frequency,'B')
            output.Data(ii,1)=datenum(str2num(tempstr{ii}(1:4)),1+6*(str2num(tempstr{ii}(7))-1),1);
        elseif strcmp(frequency,'Q')
            output.Data(ii,1)=datenum(str2num(tempstr{ii}(1:4)),1+3*(str2num(tempstr{ii}(7))-1),1);
        elseif strcmp(frequency,'M')
            output.Data(ii,1)=datenum(str2num(tempstr{ii}(1:4)),str2num(tempstr{ii}(6:7)),1);
        else
            fprinf('ERROR: I HAVE NOT CODED getIMFData FOR ANYTHING HIGHER THAN MONTHLY FREQUENCY, IF YOU NEED THIS THEN PLEASE JUST SEND ME AN EMAIL robertdkirkby@gmail.com AND I WILL IMPLEMENT IT')
        end
    end
    output.IMFcodes.info='These codes are how IMF decribe the data series. (Contents of Obs are just another copy of the data)'; % To do this for more than just the BOP & IFS I really need to just do a read of the field names and then index through them
    FullInfoNames=fieldnames(JSONdata.CompactData.DataSet.Series);
    nFields=length(FullInfoNames);
    for ii=1:nFields
        output.IMFcodes.(FullInfoNames{ii})=JSONdata.CompactData.DataSet.Series.(FullInfoNames{ii});
    end
else
    fprintf('getIMFDATA Warning: You requested some missing data, have returned a nan value. \n')
    if strcmp(database_id,'IFS') || strcmp(database_id, 'BOP') || strcmp(database_id, 'FSI')
        fprintf('    Missing: Database: %s , Series: %s , Country: %s \n', database_id_full, series_id, countrycode2L);
    elseif strcmp(database_id,'CPIS')
        fprintf('    Missing: Database: %s , Series: %s , Countries: %s and %s , Sectors: %s and %s \n', database_id_full, series_id, countrycode2L, counterpartycountrycode2L,sector_id,counterparty_id);
    elseif strcmp(database_id,'DOT')
        fprintf('    Missing: Database: %s , Series: %s , Countries: %s and %s \n', database_id_full, series_id, countrycode2L, counterpartycountrycode2L);
    end
    output.Data=nan(1,2);
end



%% GenericMetadata
% For some reason known only to IMF, it appears you can only request
% metadata for the 'annual frequency' code. So next line just imposes this
% on the metadata request (the frequency is not part of the metadata anyway).
longoptionstring(1)='A';
% Now request the metadata
JSONdata3 = webread(['http://dataservices.imf.org/REST/SDMX_JSON.svc/GenericMetadata/',database_id_full,'/',longoptionstring]);
attempts=0;
while ~isstruct(JSONdata3) && attempts<=10 % Try a few time to get the data as JSON (rather than text, which for some reason sometimes happens)
    JSONdata3 = webread(['http://dataservices.imf.org/REST/SDMX_JSON.svc/GenericMetadata/',database_id_full,'/',longoptionstring]);
    attempts=attempts+1;
    fprintf('getIMFData: the request to IMF failed to parse, reattempting (this msg is purely FYI, not indicative of eventual error) \n')
    if attempts==10 && ~isstruct(JSONdata3)
        fprintf('ERROR, getIMFData: The request to IMF keeps failing to parse, typically this is solved by just rerunning (there is no discernable reason for this error)')
    end
end
% Sort the metadata into output
output.country=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(2).ReportedAttribute(1).ReportedAttribute(1).Value.x_text;
if length(JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet)>2 % If there is no data then the following are sometimes missing
    output.series_id=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(3).ReportedAttribute(1).ReportedAttribute(1).Value.x_text;
    output.description=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(3).ReportedAttribute(2).ReportedAttribute(1).Value.x_text;
    if strcmp(database_id,'CPIS')
        output.sector=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(4).ReportedAttribute(2).ReportedAttribute(1).Value.x_text;
        output.counterpartsector=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(5).ReportedAttribute(2).ReportedAttribute(1).Value.x_text;
        output.counterpartcountry=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(6).ReportedAttribute(2).ReportedAttribute(1).Value.x_text;
    end
    if strcmp(database_id,'DOT')
        output.counterpartcountry=JSONdata3.GenericMetadata.MetadataSet.AttributeValueSet(4).ReportedAttribute(2).ReportedAttribute(1).Value.x_text;
    end
end

end