Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions include/rawtoaces/rawtoaces_core.h
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,25 @@ class SpectralSolver
std::vector<std::string>
collect_data_files( const std::string &type ) const;

/// A helper method collecting aliases of a given type from the database.
/// This function searches through the configured search directories to
/// find all make/model aliases of the specified type ("camera" is the only
/// currently supported type).
///
/// @param type type of the aliases to search for (e.g., "camera")
/// @param make_aliases output parameter referencing the map
/// to be filled with the found make aliases.
/// @param model_aliases output parameter referencing the map
/// to be filled with the found model aliases.
/// @return `true` on success.
bool collect_aliases(
const std::string &type,
std::map<std::string, std::string> &make_aliases,
std::map<
std::string,
std::map<std::string, std::pair<std::string, std::string>>>
&model_aliases ) const;

/// A helper method loading the spectral data for a file at the given path.
/// This function loads spectral data from a file, handling both absolute and relative paths.
/// For relative paths, it searches through all configured search directories.
Expand Down
165 changes: 163 additions & 2 deletions src/rawtoaces_core/rawtoaces_core.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@
// Copyright Contributors to the rawtoaces Project.

#include <rawtoaces/rawtoaces_core.h>

#include <nlohmann/json.hpp>
#include <fstream>

#include "rawtoaces_core_priv.h"
#include "mathOps.h"
#include "define.h"
Expand Down Expand Up @@ -266,6 +270,109 @@ SpectralSolver::collect_data_files( const std::string &type ) const
return result;
}

bool SpectralSolver::collect_aliases(
const std::string &type,
std::map<std::string, std::string> &make_aliases,
std::map<
std::string,
std::map<std::string, std::pair<std::string, std::string>>>
&model_aliases ) const
{
for ( const auto &directory: _search_directories )
{
std::filesystem::path path( directory );
path.append( "aliases.json" );

if ( std::filesystem::exists( path ) )
{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returning true when path doesn't exist is expected?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an optional file, it is not expected to be in every directory specified in the paths list.

// For now we consider the aliases files optional.
// In a case of error, we just skip the file and process the next
// one. We still want to report warnings to let the user a chance
// to fix broken files.

std::ifstream ifstream( path );
if ( !ifstream.is_open() )
{
std::cerr << "Warning: cannot open file " << path.string()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do show warning when you can't open the path, but we don't dow it for when it doesn't exist in a first place. expected?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expected. This path is actually the only bit which is not covered in unit tests. Same with the spectral files. May want to come up with a way to test if an existing file can't be opened, like restricting file permissions, or something. Will leave this for later, as it is not directly related to this feature.

<< "." << std::endl;
continue;
}

nlohmann::json file_data;

try
{
file_data = nlohmann::json::parse( ifstream );
}
catch ( const std::exception &error )
{
std::cerr << "Warning: JSON parsing of the alias file "
<< path.string()
<< " failed with error: " << error.what() << "."
<< std::endl;
continue;
}

if ( !file_data.contains( "data" ) )
{
std::cerr << "Warning: the alias file " << path.string()
<< " must contain the 'data' section." << std::endl;
continue;
}

const auto &mapping_data = file_data["data"];

if ( !mapping_data.contains( type ) )
{
continue;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some "skip" we show warning, for others we don't

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section is optional, being omitted from an aliases files is a normal case. We just go to the next file.

}

const auto &type_mapping = mapping_data[type];

if ( type_mapping.contains( "make" ) )
{
const auto &make_mapping = type_mapping["make"];
for ( auto &[src_make, dst_make]: make_mapping.items() )
{
if ( make_aliases.find( src_make ) == make_aliases.end() )
{
make_aliases[src_make] = dst_make;
}
}
}

if ( type_mapping.contains( "model" ) )
{
const auto &model_mapping = type_mapping["model"];
for ( auto &[src_make, models]: model_mapping.items() )
{
for ( auto &[src_model, dst]: models.items() )
{
const std::string src_make_str = src_make;
const std::string src_model_str = src_model;
if ( model_aliases.find( src_make_str ) !=
model_aliases.end() &&
model_aliases[src_make_str].find(
src_model_str ) !=
model_aliases[src_make_str].end() )
{
continue;
}

const std::string dst_make_str = dst["make"];
const std::string dst_model_str = dst["model"];
model_aliases[src_make_str][src_model_str] = {
dst_make_str, dst_model_str
};
}
}
}
}
}

return true;
}

bool SpectralSolver::load_spectral_data(
const std::string &file_path, SpectralData &out_data )
{
Expand Down Expand Up @@ -304,15 +411,69 @@ bool SpectralSolver::find_camera(
return true;
}

std::string dst_make = make;
std::string dst_model = model;

std::map<std::string, std::string> make_aliases;
std::map<
std::string,
std::map<std::string, std::pair<std::string, std::string>>>
model_aliases;

if ( collect_aliases( "camera", make_aliases, model_aliases ) )
{

bool found_make_alias = false;
for ( const auto &[alias_src_make, alias_dst_make]: make_aliases )
{
if ( is_not_equal_insensitive( make, alias_src_make ) )
continue;

dst_make = alias_dst_make;
found_make_alias = true;
break;
}

bool found_model_alias = false;
for ( const auto &[alias_src_make, alias_src_dict]: model_aliases )
{
if ( is_not_equal_insensitive( dst_make, alias_src_make ) )
continue;

for ( const auto &[alias_src_model, alias_dst_pair]:
alias_src_dict )
{
if ( is_not_equal_insensitive( model, alias_src_model ) )
continue;
dst_make = alias_dst_pair.first;
dst_model = alias_dst_pair.second;
found_model_alias = true;
break;
}

if ( found_model_alias )
break;
}

if ( found_make_alias || found_model_alias )
{
if ( !is_not_equal_insensitive( camera.manufacturer, dst_make ) &&
!is_not_equal_insensitive( camera.model, dst_model ) )
{
return true;
}
}
}

auto camera_files = collect_data_files( "camera" );

for ( const auto &camera_file: camera_files )
{
camera.load( camera_file );

if ( is_not_equal_insensitive( camera.manufacturer, make ) )
if ( is_not_equal_insensitive( camera.manufacturer, dst_make ) )
continue;
if ( is_not_equal_insensitive( camera.model, model ) )
if ( is_not_equal_insensitive( camera.model, dst_model ) )
continue;
return true;
}
Expand Down
1 change: 1 addition & 0 deletions src/rawtoaces_util/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ add_library ( ${RAWTOACES_UTIL_LIB} ${DO_SHARED}
colour_transforms.h
exiftool.cpp
exiftool.h
rawtoaces_util_priv.h

$<$<BOOL:${RTA_ENABLE_LENSFUN}>:
lens_correction.cpp
Expand Down
82 changes: 81 additions & 1 deletion src/rawtoaces_util/image_converter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1409,6 +1409,43 @@ std::vector<std::string> ImageConverter::get_supported_illuminants() const
return result;
}

/// Combine the make and model strings together, unless they are the same,
/// or the model name starts with the make followed by a whitespace.
/// "Canon" + "Canon R5" = "Canon R5"
/// "Canon R5" + "Canon R5" = "Canon R5"
/// "Canon" + "Canon_R5" = "Canon Canon_R5"
std::string
combine_make_model( const std::string &make, const std::string &model )
{
bool has_prefix = true;

if ( make.length() <= model.length() )

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am very confused about this logic. If it is correct we should leave comment here.
I've looked at the test and I still don't get it it =))
It is very specific, logic in this method. Has a lot of assumptions and build in heuristics. And it is not very clear. Comment would def. be needed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is trivial really. Skip the prefix if it matches the make name, or the make name + whitespace.
"Canon" + "Canon R5" = "Canon R5"
"Canon R5" + "Canon R5" = "Canon R5"
"Canon" + "Canon_R5" = "Canon Canon_R5"

{
size_t i;
for ( i = 0; i < make.length(); i++ )
{
if ( std::tolower( make[i] ) != std::tolower( model[i] ) )
{
has_prefix = false;
break;
}
}
if ( has_prefix && ( i < model.length() ) )
{
has_prefix = model[i] == ' ';
}
}
else
{
has_prefix = false;
}

if ( has_prefix )
return model;
else
return make + " " + model;
}

std::vector<std::string> ImageConverter::get_supported_cameras() const
{
std::vector<std::string> result;
Expand All @@ -1420,11 +1457,54 @@ std::vector<std::string> ImageConverter::get_supported_cameras() const
core::SpectralData data;
if ( data.load( file, false ) )
{
std::string name = data.manufacturer + " / " + data.model;
std::string name =
combine_make_model( data.manufacturer, data.model );
result.push_back( name );
}
}

std::map<std::string, std::string> make_aliases;
std::map<
std::string,
std::map<std::string, std::pair<std::string, std::string>>>
model_aliases;
if ( solver.collect_aliases( "camera", make_aliases, model_aliases ) )
{
std::vector<std::string> parsed_aliases;
for ( auto &[src_make, src_models]: model_aliases )
{
for ( auto &[src_model, dst]: src_models )
{
const std::string &dst_make = dst.first;
const std::string &dst_model = dst.second;
const std::string dst_make_model =
combine_make_model( dst_make, dst_model );
const std::string src_make_model =
combine_make_model( src_make, src_model );

if ( std::find(
result.begin(), result.end(), dst_make_model ) !=
result.end() )
{
const std::string entry =
src_make_model + " (alias for " + dst_make_model + ")";
parsed_aliases.push_back( entry );
}
else
{
std::cerr << "Warning: the alias " << src_make_model
<< " points to a missing camera "
<< dst_make_model << std::endl;
}
}
}

result.insert(
result.end(), parsed_aliases.begin(), parsed_aliases.end() );
}

std::sort( result.begin(), result.end() );

return result;
}

Expand Down
3 changes: 3 additions & 0 deletions src/rawtoaces_util/rawtoaces_util_priv.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ std::vector<std::string>
database_paths( const std::string &override_path = "" );
void fix_metadata( OIIO::ImageSpec &spec );

std::string
combine_make_model( const std::string &make, const std::string &model );

bool fetch_missing_metadata(
const std::string &input_path,
const ImageConverter::Settings &settings,
Expand Down
Loading
Loading