assertEquals(firstSubQueue.getJobQueueInfo().getQueueName(), "first"); assertEquals(firstSubQueue.getJobQueueInfo().getQueueState(), "running"); assertEquals(firstSubQueue.getJobQueueInfo().getSchedulingInfo(), Set<String> queueJobQueueInfos = new HashSet<String>(); for(JobQueueInfo jobInfo : manager.getJobQueueInfos()){ queueJobQueueInfos.add(jobInfo.getQueueName()); rootJobQueueInfos.add(queue.getJobQueueInfo().getQueueName()); manager.getJobQueueInfoMapping().get("first").getQueueName(), "first");
private void updateSchedulerProps(JobQueueInfo jqi) { LOG.info("Updating properties for SchedulerQueue " + jqi.getQueueName()); LOG.info("Putting " + jqi.getProperties() + " in " + jqi.getQueueName()); schedulerPropsMap.put(jqi.getQueueName(), jqi.getProperties()); for (JobQueueInfo child : jqi.getChildren()) { updateSchedulerProps(child); } }
private static String getTree(String parent, List<JobQueueInfo> children) { StringBuilder str = new StringBuilder(); if (children == null) { return ""; } for (JobQueueInfo queueInfo : children) { String variableName = StringUtils.replaceChars(queueInfo.getQueueName(), ":-*+#.^", "_______"); String label = queueInfo.getQueueName().split(":")[queueInfo .getQueueName().split(":").length - 1]; str.append(String.format( "var %sTreeNode = new YAHOO.widget.MenuNode(\"%s\", %s, false);\n", variableName, label, parent)); str.append(String.format("%sTreeNode.data=\"%s\";\n", variableName, queueInfo.getSchedulingInfo().replaceAll("\n", "<br/>"))); str.append(String.format("%sTreeNode.name=\"%s\";\n", variableName, queueInfo.getQueueName())); str.append(getTree(variableName + "TreeNode", queueInfo.getChildren())); } return str.toString(); }
void printJobQueueInfo(JobQueueInfo jobQueueInfo, Writer writer) throws IOException { if (jobQueueInfo == null) { writer.write("No queue found.\n"); writer.flush(); return; } writer.write(String.format("Queue Name : %s \n", jobQueueInfo.getQueueName())); writer.write(String.format("Queue State : %s \n", jobQueueInfo.getQueueState())); writer.write(String.format("Scheduling Info : %s \n", jobQueueInfo.getSchedulingInfo())); List<JobQueueInfo> childQueues = jobQueueInfo.getChildren(); if (childQueues != null && childQueues.size() > 0) { writer.write(String.format("Child Queues : ")); for (int i = 0; i < childQueues.size(); i++) { JobQueueInfo childQueue = childQueues.get(i); writer.write(String.format("%s", childQueue.getQueueName())); if (i != childQueues.size() - 1) { writer.write(String.format(", ")); } } writer.write("\n"); } writer.write(String.format("======================\n")); writer.flush(); }
/** * Method used to display the list of the JobQueues registered * with the {@link QueueManager} * * @throws IOException */ private void displayQueueList() throws IOException { JobQueueInfo[] queues = jc.getQueues(); for (JobQueueInfo queue : queues) { String schedInfo = queue.getSchedulingInfo(); if(schedInfo.trim().equals("")){ schedInfo = "N/A"; } System.out.printf("Queue Name : %s \n", queue.getQueueName()); System.out.printf("Scheduling Info : %s \n",queue.getSchedulingInfo()); } }
/** * Method used to display the list of the JobQueues registered * with the {@link QueueManager} * * @throws IOException */ private void displayQueueList() throws IOException { JobQueueInfo[] queues = jc.getQueues(); for (JobQueueInfo queue : queues) { String schedInfo = queue.getSchedulingInfo(); if(schedInfo.trim().equals("")){ schedInfo = "N/A"; } System.out.printf("Queue Name : %s \n", queue.getQueueName()); System.out.printf("Scheduling Info : %s \n",queue.getSchedulingInfo()); } }
@SuppressWarnings("deprecation") void printJobQueueInfo(JobQueueInfo jobQueueInfo, Writer writer, String prefix) throws IOException { if (jobQueueInfo == null) { writer.write("No queue found.\n"); writer.flush(); return; } writer.write(String.format(prefix + "======================\n")); writer.write(String.format(prefix + "Queue Name : %s \n", jobQueueInfo.getQueueName())); writer.write(String.format(prefix + "Queue State : %s \n", jobQueueInfo.getQueueState())); writer.write(String.format(prefix + "Scheduling Info : %s \n", jobQueueInfo.getSchedulingInfo())); List<JobQueueInfo> childQueues = jobQueueInfo.getChildren(); if (childQueues != null && childQueues.size() > 0) { for (int i = 0; i < childQueues.size(); i++) { printJobQueueInfo(childQueues.get(i), writer, " " + prefix); } } writer.flush(); }
@SuppressWarnings("deprecation") void printJobQueueInfo(JobQueueInfo jobQueueInfo, Writer writer, String prefix) throws IOException { if (jobQueueInfo == null) { writer.write("No queue found.\n"); writer.flush(); return; } writer.write(String.format(prefix + "======================\n")); writer.write(String.format(prefix + "Queue Name : %s \n", jobQueueInfo.getQueueName())); writer.write(String.format(prefix + "Queue State : %s \n", jobQueueInfo.getQueueState())); writer.write(String.format(prefix + "Scheduling Info : %s \n", jobQueueInfo.getSchedulingInfo())); List<JobQueueInfo> childQueues = jobQueueInfo.getChildren(); if (childQueues != null && childQueues.size() > 0) { for (int i = 0; i < childQueues.size(); i++) { printJobQueueInfo(childQueues.get(i), writer, " " + prefix); } } writer.flush(); }
@SuppressWarnings("deprecation") void printJobQueueInfo(JobQueueInfo jobQueueInfo, Writer writer, String prefix) throws IOException { if (jobQueueInfo == null) { writer.write("No queue found.\n"); writer.flush(); return; } writer.write(String.format(prefix + "======================\n")); writer.write(String.format(prefix + "Queue Name : %s \n", jobQueueInfo.getQueueName())); writer.write(String.format(prefix + "Queue State : %s \n", jobQueueInfo.getQueueState())); writer.write(String.format(prefix + "Scheduling Info : %s \n", jobQueueInfo.getSchedulingInfo())); List<JobQueueInfo> childQueues = jobQueueInfo.getChildren(); if (childQueues != null && childQueues.size() > 0) { for (int i = 0; i < childQueues.size(); i++) { printJobQueueInfo(childQueues.get(i), writer, " " + prefix); } } writer.flush(); }
@SuppressWarnings("deprecation") void printJobQueueInfo(JobQueueInfo jobQueueInfo, Writer writer, String prefix) throws IOException { if (jobQueueInfo == null) { writer.write("No queue found.\n"); writer.flush(); return; } writer.write(String.format(prefix + "======================\n")); writer.write(String.format(prefix + "Queue Name : %s \n", jobQueueInfo.getQueueName())); writer.write(String.format(prefix + "Queue State : %s \n", jobQueueInfo.getQueueState())); writer.write(String.format(prefix + "Scheduling Info : %s \n", jobQueueInfo.getSchedulingInfo())); List<JobQueueInfo> childQueues = jobQueueInfo.getChildren(); if (childQueues != null && childQueues.size() > 0) { for (int i = 0; i < childQueues.size(); i++) { printJobQueueInfo(childQueues.get(i), writer, " " + prefix); } } writer.flush(); }
private QueueInfo[] getQueueInfoArray(JobQueueInfo[] queues) throws IOException { for (JobQueueInfo queue : queues) { queue.setJobStatuses(getJobsFromQueue(queue.getQueueName())); for(JobQueueInfo childqueue : queue.getChildren()){ childqueue.setJobStatuses(getJobsFromQueue(childqueue.getQueueName())); } } return queues; }
@Test public void testQueueOrdering() throws Exception { // create some sample queues in a hierarchy.. JobQueueInfo[] roots = new JobQueueInfo[2]; roots[0] = new JobQueueInfo("q1", "q1 scheduling info"); roots[1] = new JobQueueInfo("q2", "q2 scheduling info"); List<JobQueueInfo> children = new ArrayList<JobQueueInfo>(); children.add(new JobQueueInfo("q1:1", null)); children.add(new JobQueueInfo("q1:2", null)); roots[0].setChildren(children); // test dfs ordering JobQueueClient client = new JobQueueClient(new JobConf()); List<JobQueueInfo> allQueues = client.expandQueueList(roots); assertEquals(4, allQueues.size()); assertEquals("q1", allQueues.get(0).getQueueName()); assertEquals("q1:1", allQueues.get(1).getQueueName()); assertEquals("q1:2", allQueues.get(2).getQueueName()); assertEquals("q2", allQueues.get(3).getQueueName()); }
/** * Method used to display information pertaining to a Single JobQueue * registered with the {@link QueueManager}. Display of the Jobs is * determine by the boolean * * @throws IOException */ private void displayQueueInfo(String queue, boolean showJobs) throws IOException { JobQueueInfo schedInfo = jc.getQueueInfo(queue); if (schedInfo == null) { System.out.printf("Queue Name : %s has no scheduling information \n", queue); } else { System.out.printf("Queue Name : %s \n", schedInfo.getQueueName()); System.out.printf("Scheduling Info : %s \n",schedInfo.getSchedulingInfo()); } if (showJobs) { System.out.printf("Job List\n"); JobStatus[] jobs = jc.getJobsFromQueue(queue); if (jobs == null) jobs = new JobStatus[0]; jc.displayJobList(jobs); } }
@Override public synchronized void start() throws IOException { super.start(); QueueManager queueManager = taskTrackerManager.getQueueManager(); String queueName = queueManager.getJobQueueInfos()[0].getQueueName(); queueManager.setSchedulerInfo(queueName ,"Maximum Tasks Per Job :: " + String.valueOf(maxTasksPerJob)); }
@Override public synchronized void start() throws IOException { super.start(); QueueManager queueManager = taskTrackerManager.getQueueManager(); String queueName = queueManager.getJobQueueInfos()[0].getQueueName(); queueManager.setSchedulerInfo(queueName ,"Maximum Tasks Per Job :: " + String.valueOf(maxTasksPerJob)); }
/** * Method used to display information pertaining to a Single JobQueue * registered with the {@link QueueManager}. Display of the Jobs is * determine by the boolean * * @throws IOException */ private void displayQueueInfo(String queue, boolean showJobs) throws IOException { JobQueueInfo schedInfo = jc.getQueueInfo(queue); if (schedInfo == null) { System.out.printf("Queue Name : %s has no scheduling information \n", queue); } else { System.out.printf("Queue Name : %s \n", schedInfo.getQueueName()); System.out.printf("Scheduling Info : %s \n",schedInfo.getSchedulingInfo()); } if (showJobs) { System.out.printf("Job List\n"); JobStatus[] jobs = jc.getJobsFromQueue(queue); if (jobs == null) jobs = new JobStatus[0]; jc.displayJobList(jobs); } }
@Override public synchronized void start() throws IOException { super.start(); QueueManager queueManager = taskTrackerManager.getQueueManager(); String queueName = queueManager.getJobQueueInfos()[0].getQueueName(); queueManager.setSchedulerInfo(queueName ,"Maximum Tasks Per Job :: " + String.valueOf(maxTasksPerJob)); }
@Override public QueueInfo getQueue(String queue) throws IOException { JobQueueInfo jqueue = queueManager.getJobQueueInfo(queue); if (jqueue != null) { jqueue.setJobStatuses(getJobsFromQueue(jqueue.getQueueName())); } return jqueue; }
@Test public void testMissingConfigFile() throws Exception { deleteQueuesConfigFile(); // deletes file try { new QueueManager(QUEUES_CONFIG_FILE_PATH, true); fail("Should throw an exception for missing file when " + "explicitly passed."); } catch (RuntimeException re) { } // If we just want to pick up the queues from the class loader // it should fall through to the default. The class loader is set to // load CONFIG for the "mapred-queues.xml" resource, but it's missing // so should fall through to mapred-queues-default.xml QueueManager qm = new QueueManager(); List<JobQueueInfo> rootQueues = qm.getRoot().getJobQueueInfo().getChildren(); assertEquals(1, rootQueues.size()); assertEquals("default", rootQueues.get(0).getQueueName()); }
private void checkHierarchy(Queue queue, QueueManager queueManager) { JobQueueInfo jobQueueInfo = queueManager.getJobQueueInfo(queue.getName()); assertEquals(queue.getName(),jobQueueInfo.getQueueName()); assertEquals(queue.getState(),jobQueueInfo.getState()); if (queue.getChildren() !=null && queue.getChildren().size() > 0) { for (Queue childQueue : queue.getChildren()) { checkHierarchy(childQueue, queueManager); } } }