ClearCase repositories not detected

[AmeyaBhujbal]

I have both perforce as well as ClearCase repositories in my OpenGrok. There are total 12 repo (8 perforce and 4 clearcase)
Everything works fine with perforce, but 4 clearcase repositories are not getting detected and the historycache is not generating for the same, hence History & Annotate links are not working for Clearcase repositories.

Indexing command:
${JAVA} ${JAVA_OPTS} -Djava.util.logging.config.file=${PROGDIR}/etc/logging.properties
-jar ${PROGDIR}/dist/lib/opengrok.jar
-c /usr/local/bin/ctags
-s ${PROGDIR}/src -d ${PROGDIR}/data -H -P -S -G
-W ${PROGDIR}/etc/configuration.xml -U http://host:8080/opengrok >> ${LOG_FILE} 2> ${LOG_FILE}.err

OpenGrok log (not scanning 4 clearcase repositories):
Done scanning for repositories, found 8 repositories

OpenGrok log (clearcase commands):
Executing command [cleartool -version] in directory /home/webapp
Executing command [cleartool lsvob -s] in directory /home/webapp

Catalina.out log:
org.opengrok.indexer.util.Statistics.logIt done invalidating 8 repositories (took 4.315 seconds) [no clearcase repo was invalidated]
org.opengrok.suggest.SuggesterProjectData.initFields Fields [hist] will be ignored because they were not found in index directory /path/to/index/dir/ [dir is present in the path]

I am using OpenGrok 1.7.2,
Apache Tomcat 10.0.5,
JDK 11.0.7,
Linux/unix system.

[vladak]

Some observations:

As you noticed, a directory needs to be identified as a Clearcase repository first before history cache can be generated for it.

There was an attempt to execute the cleartool command however it is not clear (sic!) if it was successful. Does the log contain other relevant entries ? What happens if you run these commands in a terminal ?

For a directory to be detected as Clearcase repository one of the conditions needs to be satisfied:

opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/history/ClearCaseRepository.java

Lines 198 to 222 in 9e27dd5

	boolean isRepositoryFor(File file, CommandTimeoutType cmdType) {
	// if the parent contains a file named "[.]view.dat" or
	// the parent is named "vobs" or the canonical path
	// is found in "cleartool lsvob -s"
	File fWindows = new File(file, "view.dat");
	File fUnix = new File(file, ".view.dat");
	if (fWindows.exists() || fUnix.exists()) {
	return true;
	} else if (file.isDirectory() && file.getName().equalsIgnoreCase("vobs")) {
	return true;
	} else if (isWorking()) {
	try {
	String canonicalPath = file.getCanonicalPath();
	for (String vob : getAllVobs()) {
	if (canonicalPath.equalsIgnoreCase(vob)) {
	return true;
	}
	}
	} catch (IOException e) {
	LOGGER.log(Level.WARNING,
	"Could not get canonical path for \"" + file + "\"", e);
	}
	}
	return false;
	}

[vladak]

Also, similar thing as with #3576: if the cleartool command is not available in the usual locations ($PATH) then it is necessary to point the Indexer to the actual location with a system property: https://github.com/oracle/opengrok/wiki/Indexer-configuration#java-system-properties

[AmeyaBhujbal]

log of cleartool command:

2021-05-07 05:30:19.655+0530 FINE t20 Statistics.logIt: Finished command [cleartool -version] in directory /home/webapp with exit code 0 (took 211 ms)
259 2021-05-07 05:30:19.655+0530 FINE t20 Executor.exec: Finished command [cleartool -version] in directory /home/webapp with exit code 0
260 2021-05-07 05:30:19.656+0530 FINE t21 Statistics.logIt: Finished command [cleartool -version] in directory /home/webapp with exit code 0 (took 209 ms)
261 2021-05-07 05:30:19.656+0530 FINE t19 Statistics.logIt: Finished command [cleartool -version] in directory /home/webapp with exit code 0 (took 207 ms)
262 2021-05-07 05:30:19.656+0530 FINE t18 Statistics.logIt: Finished command [cleartool -version] in directory /home/webapp with exit code 0 (took 212 ms)
263 2021-05-07 05:30:19.657+0530 FINE t19 Executor.exec: Finished command [cleartool -version] in directory /home/webapp with exit code 0
264 2021-05-07 05:30:19.656+0530 FINE t21 Executor.exec: Finished command [cleartool -version] in directory /home/webapp with exit code 0
265 2021-05-07 05:30:19.657+0530 FINE t18 Executor.exec: Finished command [cleartool -version] in directory /home/webapp with exit code 0
2021-05-07 05:30:25.199+0530 FINE t21 Statistics.logIt: Finished command [cleartool lsvob -s] in directory /home/webapp with exit code 0 (took 5.499 seconds)
271 2021-05-07 05:30:25.200+0530 FINE t21 Executor.exec: Finished command [cleartool lsvob -s] in directory /home/webapp with exit code 0

and if I run cleartool commands in the terminal, it works fine.

[vladak]

Looks like the cleartool runs fine in the indexer context. I assume isWorking() returns true. So the likely reason why the Clearcase repositories are not detected is because they are not listed in the output of cleartool lsvob -s or because the .view.dat file is not present.

[AmeyaBhujbal]

We are using clearcase version 9 and its a DYNAMIC VIEW not SNAPSHOT VIEW. (Linux System)

[vladak]

I never worked with Clearcase before so this is something you will have to figure out.