We study the computation of the diameter and radius under the rectilinear link distance within a rectilinear polygonal domain of n vertices and h holes. We introduce a graph of oriented distances to encode the distance between pairs of points of the domain. This helps us transform the problem so that we can search through the candidates more efficiently. Our algorithm computes both the diameter and the radius in O(min(n^ω, n² + nhlog h + χ²)) time, where ω < 2.373 denotes the matrix multiplication exponent and χ ∈ Ω(n) ∩ O(n²) is the number of edges of the graph of oriented distances. We also provide an alternative algorithm for computing the diameter that runs in O(n² log n) time.