Dynamic two-way references.

[rlopez-plenty]

Hello Community.

Our goal is that any #Object.mappings defined in one spot should show up as the inverse in the second #Object.mappings on compilation such that both Objects are aware that they are mapped to each other, but we only have to establish the relationship in one spot for simplicity. While it totally works to define the mappings in two places, we'd like to be more dynamic in generating these mappings to save time. The closest we've been able to get is to utilize for loops to go through every #Object.mappings element and try to generate the inverse, but we are struggling to modify the referenced object. In our example, this would be iterating through Beta.Beta1.mappings to create Alpha.Alpha1.mappings that point back to Beta1. Another potential solution we've come up with is breaking up the to: element in mappings into separate variables, which somewhat works but ends up with extraneous output (and is a bit verbose). An added element of difficulty is that #Objects can be nested in other #Objects, which potentially introduces some element of infinite looping issues?

// This final output matches what we are hoping to achieve
// but is hardcoded to work, need a dynamic solution
TEMP: {
    // If we define a mapping here, the inverse should show up at that mapping
    // ie, Alpha.Alpha1.id should now map to Beta.Beta1.id
    // (are there potential recursive infinite loop problems?)
    #Object: {...}
	Beta: #Object & {
		Beta1: #Object & {
		    id: "B1234"
		    // would like to keep string1, string2, and mappedId hidden
		    // or ideally, just use the "to" definition to access the right values
			mappings: [{to: Alpha.Alpha1.id, string1: "Alpha", string2: "Alpha1", mappedId: id}]
		}
	}
	// This code chunk would ideally live under Beta1 or Beta1.mappings
	let temp_mappings = Beta.Beta1.mappings
	for mapping in temp_mappings {
	    // Ideally we would have access to Alpha.Alpha1.id structure here
	    // without string1 and string2
        "\(mapping.string1)": {
            "\(mapping.string2)": {
                mappings: [{to: mapping.mappedId}] //just id if inside Beta1
            }
        }
    }
    // Object we want to map back to Beta.Beta1.id
	Alpha: #Object & {
		Alpha1: #Object & {
			id:    "A1234"
			//mappings: [{to: Beta.Beta1.id}] //we want to generate this dynamically
		}
	}
}

and

{
    "TEMP": {
        "Beta": {
            "Beta1": {
                "id": "B1234",
                "mappings": [
                    {
                        "to": "A1234",
                        "string1": "Alpha", // ideally doesn't show up in final compilation
                        "string2": "Alpha1", // ideally doesn't show up in final compilation
                        "mappedId": "B1234" // ideally doesn't show up in final compilation
                    }
                ]
            }
        },
        "Alpha": {
            "Alpha1": {
                "id": "A1234",
                "mappings": [
                    {
                        "to": "B1234"
                    }
                ]
            }
        }
    }
}

With the ideal scenario being something like:

TEMP: {
    #Object: {...
    // some for loop here to create inverse mappings 
    // so all Objects know which objects they map to and which objects map to them    
    }
	Beta: #Object & {
		Beta1: #Object & {
		    id: "B1234"
			mappings: [{to: Alpha.Alpha1.id}]
		}
	}
    // Object we want to map back to Beta.Beta1.id
	Alpha: #Object & {
		Alpha1: #Object & {
			id:    "A1234"
		}
	}
}

With this output:

{
    "TEMP": {
        "Beta": {
            "Beta1": {
                "id": "B1234",
                "mappings": [
                    {
                        "to": "A1234"
                    }
                ]
            }
        },
        "Alpha": {
            "Alpha1": {
                "id": "A1234"
                "mappings": [
                    {
                        "to": "B1234"
                    }
                ]
            }
        }
    }
}

Thanks in advance!

[mpvl]

In principle you should be able to map one object to another without getting into cycle issues. But there is an issue with comprehensions and spurious cycles, see #779. Adding new fields to a map would be okay as long as the modified fields would not influence the comprehension, which is the case here.

For now, if it is workable for you to define a duplicate object with the expanded results, you could write:

objects: foo: bar: {
	id: "bar1"
	mappings: [{to: "quux1"}]
}

objects: qux: quux: {
	id: "quux1"
}

let lookup = {
	for string1, x in objects
	for string2, y in x {
		"\(y.id)": [string1, string2]
	}
}

objects2: objects

objects2: {
	for x in objects
	for y in x
	if y.mappings != _|_
	for m in y.mappings
	let path = lookup[m.to] {
		"\(path[0])": "\(path[1])": mappings: [
			{to: y.id}]
	}
}

[mpvl]

And feel free to comment on the related issue if this is blocking you.

[mpvl]

Ah, you also probably want mappings to be a map, so the results can be merged, at least as an intermediate value.

[rlopez-plenty]

Thanks a lot for your quick response Marcel, your solution worked! This is a really good approach. I really appreciate your help.

Now, we have another scenario where we want to ask:
Imagine now that we have different levels of depth (we have about 6 levels) and we have mappings at different levels, for example this case, I added a new mapping from Beta1 to Alpha and in that order we also want to generate that reverse mapping in Alpha to Beta1, do you have any thoughts?

// This final output matches what we are hoping to achieve
// but is hardcoded to work, need a dynamic solution
_TEMP: {
    // If we define a mapping here, the inverse should show up at that mapping
    // ie, Alpha.Alpha1.id should now map to Beta.Beta1.id
    // (are there potential recursive infinite loop problems?)
    #Object: {...}
	Beta: #Object & { 
		Beta1: #Object & {
		    id: "B1234"
		    // would like to keep string1, string2, and mappedId hidden
		    // or ideally, just use the "to" definition to access the right values
			mappings: [{to: Alpha.Alpha1.id},{to: Alpha.id}]
		}
	}
    // Object we want to map back to Beta.Beta1.id
	Alpha: #Object & {
		id: "A1"
		//mappings: [{to: Beta.Beta1.id}] //we also want to generate this dynamically
		Alpha1: #Object & {
			id:    "A1234"
			//mappings: [{to: Beta.Beta1.id}] //we want to generate this dynamically
		}
	}
}
let lookup = {
	for string1, x in _TEMP
	for string2, y in x {
		"\(y.id)": [string1, string2]
	}
}
TEMP2: _TEMP
TEMP2: {
  for x in _TEMP {
    for y in x {
      if y.mappings != _|_ {
        for m in y.mappings {
          let path = lookup[m.to]
             "\(path[0])": "\(path[1])": mappings: [{to: y.id}]}
          }
        }
    }
}

[mpvl]

This is a nice case to test against the query proposal, btw. With the query proposal implemented and the for cycle bug fix, this could be written as:

// Define mappings to be an associative array keyed on 'to':
objects: [string]: [string]: mappings: [@.to]: {to: string}

objects: {
	// For all mappings with a concrete 'to' field:
	for To in objects.[_].From=[_].mappings.[_].to {
		// match the fields with the id set to this value and add the inverse mapping.
		[string]: [string: {id: To }]: mappings: [{to: From.id}]
	}
}

The proposed associative array feature allows one to interpret lists as maps, keyed on some string derived from the list elements.

One would have to separately enforce that references exist and are unique.

[cueckoo]

This discussion has been migrated to cue-lang/cue#907.

For more details about CUE's migration to a new home, please see cue-lang/cue#1078.