in the following my rx0 is set to 0.2 and my rx1 never exceeded 7
Pyroms bathy smoother:
(Zeta are better for visualizing this, ranges are from 2m to -2m which is very unrealistic considering my init file is from -1m to 1m)
smoothing_Positive_rx0:
effect 1 hour after model started

Produced a bathymetry with severe HPGE, the r-value matrix shows that it corresponds to values of rx0 = 0.2 along the edge of steep areas. (max zeta about 8m)
smoothing_Negative_rx0:

Produced a bathymetry with less servere HPGE, it smoothed the bathymetry to a more unrealistic shape , with pyramid shaped edges (max zeta less than 3m)
The bathymetry:

Can clearly see that it smoothed the edges to pyramids, even with that unrealistic change the result isnt quite as good as I thought
Pyroms LP based Smoother:
The heuristic method: LP_smoothing_rx0_heuristic took 16 Hours to finish on my grid (1100 * 1250), which is ridiculous as some article rated it to be less than an hour vs the non heuristic method taking days to finish.
I haven't tested this setup yet and I will update this post asap, I think it would be similar to the roms_tools based smoothing mentioned below as it didnt made a big change to the bathymetry as shown in the r-value matrix
Roms_tools based smoothing:
I used the julia port of the matlab code ...
https://alexander-barth.github.io/ROMS.jl/dev/
http://forge.ipsl.jussieu.fr/roms_locea ... id.m?rev=1
The bathymetry is not unrealistic but the result is strictly worse than the Negative x0

Note : rx1 is 7 except the smoothing_Negative_rx0 case which is 5.
Hoping to have some advice on bathymetry smoothing methods that's available, or some strategies I can employ using the current ones I have. Should I try to reduce my vertical layers or keep on smoothing the bathymetry ... (it is about 35 right now and the maximum rx1 is about 5 to 7)
I will have to add that I am using python scripts only, I basically took the matlab files to make the forcing, clm , bdy , ini files and rewrote them (to a level that it can be considered original code) into python versions. And I feel that non-matlab based methods are severely under represented, I had to heavily modify some of pyroms' code due to either updates in packages or error in the original code as its so old ...
here's a hidden gem I've found that's not been mentioned in this forum and isn't more than a quarter of my age with unfinished on its README: ( I discovered this yesterday and am surprised that its active )
Model 2 ROMS :
https://github.com/trondkr/model2roms
Thanks for reading and waiting for your reply
