this.addStorageDir(new StorageDirectory(new File(dirName.getPath()), dirType, this.addStorageDir(new StorageDirectory(new File(dirName.getPath()), NameNodeDirType.EDITS, sharedEditsDirs.contains(dirName)));
StorageDirectory dnSdStorage = new StorageDirectory(new File(dnRoot)); File dnPrevDir = dnSdStorage.getPreviousDir();
StorageDirectory bpSd = new StorageDirectory(bpRoot);
/** * Create a <code>version</code> file for namenode inside the specified parent * directory. If such a file already exists, it will be overwritten. * The given version string will be written to the file as the layout * version. None of the parameters may be null. * * @param parent directory where namenode VERSION file is stored * @param version StorageInfo to create VERSION file from * @param bpid Block pool Id * * @return the created version file */ public static File[] createNameNodeVersionFile(Configuration conf, File[] parent, StorageInfo version, String bpid) throws IOException { Storage storage = new NNStorage(conf, Collections.<URI>emptyList(), Collections.<URI>emptyList()); storage.setStorageInfo(version); File[] versionFiles = new File[parent.length]; for (int i = 0; i < parent.length; i++) { versionFiles[i] = new File(parent[i], "VERSION"); StorageDirectory sd = new StorageDirectory(parent[i].getParentFile()); storage.writeProperties(versionFiles[i], sd); } return versionFiles; }
void finalizedUpgrade(int namespaceId) throws IOException { // To handle finalizing a snapshot taken at datanode level while // upgrading to federation, if datanode level snapshot previous exists, // then finalize it. Else finalize the corresponding BP. for (StorageDirectory sd : storageDirs) { File prevDir = sd.getPreviousDir(); File curDir = sd.getCurrentDir(); NameSpaceSliceStorage nsStorage = nsStorageMap.get(namespaceId); File nsRoot = nsStorage.getNsRoot(namespaceId, curDir); StorageDirectory nsSd = new StorageDirectory(nsRoot); if (prevDir.exists() && nsSd.getPreviousDir().exists()) { throw new IOException("Top level and NS level previous directories" + " cannot co-exist"); } if (prevDir.exists()) { // data node level storage finalize doFinalize(sd); } else { // Name Space storage finalize using specific namespaceId nsStorage.doFinalize(curDir); } } }
private static void createEmptyInProgressEditLog(MiniDFSCluster cluster, NameNode nn, boolean writeHeader) throws IOException { long txid = nn.getNamesystem().getEditLog().getLastWrittenTxId(); URI sharedEditsUri = cluster.getSharedEditsDir(0, 1); File sharedEditsDir = new File(sharedEditsUri.getPath()); StorageDirectory storageDir = new StorageDirectory(sharedEditsDir); File inProgressFile = NameNodeAdapter.getInProgressEditsFile(storageDir, txid + 1); assertTrue("Failed to create in-progress edits file", inProgressFile.createNewFile()); if (writeHeader) { DataOutputStream out = new DataOutputStream(new FileOutputStream( inProgressFile)); EditLogFileOutputStream.writeHeader( NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION, out); out.close(); } }
/** * @param conf Configuration object * @param logDir the path to the directory in which data will be stored * @param errorReporter a callback to report errors * @throws IOException */ protected JNStorage(Configuration conf, File logDir, StartupOption startOpt, StorageErrorReporter errorReporter) throws IOException { super(NodeType.JOURNAL_NODE); sd = new StorageDirectory(logDir); this.addStorageDir(sd); this.fjm = new FileJournalManager(conf, sd, errorReporter); analyzeAndRecoverStorage(startOpt); }
/** * @param conf Configuration object * @param logDir the path to the directory in which data will be stored * @param errorReporter a callback to report errors * @throws IOException */ protected JNStorage(Configuration conf, File logDir, StartupOption startOpt, StorageErrorReporter errorReporter) throws IOException { super(NodeType.JOURNAL_NODE); sd = new StorageDirectory(logDir); this.addStorageDir(sd); this.fjm = new FileJournalManager(conf, sd, errorReporter); analyzeAndRecoverStorage(startOpt); }
public static void createBlockPoolVersionFile(File bpDir, StorageInfo version, String bpid) throws IOException { // Create block pool version files if (DataNodeLayoutVersion.supports( LayoutVersion.Feature.FEDERATION, version.layoutVersion)) { File bpCurDir = new File(bpDir, Storage.STORAGE_DIR_CURRENT); BlockPoolSliceStorage bpStorage = new BlockPoolSliceStorage(version, bpid); File versionFile = new File(bpCurDir, "VERSION"); StorageDirectory sd = new StorageDirectory(bpDir); bpStorage.writeProperties(versionFile, sd); } }
/** * Format a namespace slice storage. * @param dnCurDir DataStorage current directory * @param nsInfo the name space info * @throws IOException Signals that an I/O exception has occurred. */ void format(File dnCurDir, NamespaceInfo nsInfo) throws IOException { File curNsDir = getNsRoot(namespaceID, dnCurDir); StorageDirectory nsSdir = new StorageDirectory(curNsDir); format(nsSdir, nsInfo); }
/** * Format a block pool slice storage. * @param dnCurDir DataStorage current directory * @param nsInfo the name space info * @throws IOException Signals that an I/O exception has occurred. */ void format(File dnCurDir, NamespaceInfo nsInfo) throws IOException { File curBpDir = getBpRoot(nsInfo.getBlockPoolID(), dnCurDir); StorageDirectory bpSdir = new StorageDirectory(curBpDir); format(bpSdir, nsInfo); }
/** Check whether the root is a valid BlockPoolSlice storage. */ private static void checkDir(File root, String bpid) { Storage.StorageDirectory sd = new Storage.StorageDirectory(root); File bpRoot = new File(sd.getCurrentDir(), bpid); Storage.StorageDirectory bpSd = new Storage.StorageDirectory(bpRoot); assertTrue(bpSd.getRoot().isDirectory()); assertTrue(bpSd.getCurrentDir().isDirectory()); assertTrue(bpSd.getVersionFile().isFile()); }
/** Check whether the path is a valid DataNode data directory. */ private static void checkDir(File dataDir) { Storage.StorageDirectory sd = new Storage.StorageDirectory(dataDir); assertTrue(sd.getRoot().isDirectory()); assertTrue(sd.getCurrentDir().isDirectory()); assertTrue(sd.getVersionFile().isFile()); }
private static Storage.StorageDirectory createStorageDirectory(File root) { Storage.StorageDirectory sd = new Storage.StorageDirectory(root); DataStorage.createStorageID(sd, false); return sd; }
/** * Format a block pool slice storage. * @param dnCurDir DataStorage current directory * @param nsInfo the name space info * @throws IOException Signals that an I/O exception has occurred. */ void format(File dnCurDir, NamespaceInfo nsInfo) throws IOException { File curBpDir = getBpRoot(nsInfo.getBlockPoolID(), dnCurDir); StorageDirectory bpSdir = new StorageDirectory(curBpDir); format(bpSdir, nsInfo); }
StubBlockPoolSliceStorage(int namespaceID, String bpID, long cTime, String clusterId) { super(namespaceID, bpID, cTime, clusterId); addStorageDir(new StorageDirectory(new File("/tmp/dontcare/" + bpID))); assertThat(storageDirs.size(), is(1)); } }
List<Callable<StorageDirectory>> callables, Configuration conf) throws IOException { StorageDirectory sd = new StorageDirectory( nsInfo.getBlockPoolID(), null, true, location); try {
private StorageDirectory loadStorageDirectory(DataNode datanode, NamespaceInfo nsInfo, StorageLocation location, StartupOption startOpt, List<Callable<StorageDirectory>> callables) throws IOException { StorageDirectory sd = new StorageDirectory(null, false, location); try { StorageState curState = sd.analyzeStorage(startOpt, this, true);
/** * @param conf Configuration object * @param logDir the path to the directory in which data will be stored * @param errorReporter a callback to report errors * @throws IOException */ protected JNStorage(Configuration conf, File logDir, StartupOption startOpt, StorageErrorReporter errorReporter) throws IOException { super(NodeType.JOURNAL_NODE); sd = new StorageDirectory(logDir); this.addStorageDir(sd); this.fjm = new FileJournalManager(conf, sd, errorReporter); analyzeAndRecoverStorage(startOpt); }
/** * Format a block pool slice storage. * @param dnCurDir DataStorage current directory * @param nsInfo the name space info * @throws IOException Signals that an I/O exception has occurred. */ void format(File dnCurDir, NamespaceInfo nsInfo) throws IOException { File curBpDir = getBpRoot(nsInfo.getBlockPoolID(), dnCurDir); StorageDirectory bpSdir = new StorageDirectory(curBpDir); format(bpSdir, nsInfo); }