Discussions  >  CollabNet Subversion Admins  >  Re: Loading huge dump fails

Hide all messages in topic

All messages in topic

Re: Loading huge dump fails Re: Loading huge dump fails

Author jeyanthan (OCN Member)
Full name Jeyanthan I
Date 2012-03-19 10:08:15 PDT
Message > PS : I am subscribed to this discussion forum. However, I am not receiving any e-mails.
>

You should get mail notifications from now on. I saw your username was
not set to 'Active' and I made it Active now.

--
Regards,
Jeyanthan

Re: Loading huge dump fails Re: Loading huge dump fails

Author markphip (OCN Member)
Full name Mark Phippard
Date 2012-03-19 09:44:30 PDT
Message On Mar 19, 2012, at 12:39 PM, Karthikeyan Sundharrajan wrote:

> Thanks a lot, it worked ! Since Edge provides an option to dump in zip format, I thought it would work for Load as well. It appears, my assumption was wrong.
> So, dumped the repo in .dump format and loaded them successfully.

Edge does support .zip files but you need to be on the 2.2.1 or 2.3.0 release for it to work with zip files that are > 4GB.

Mark

RE: Re: Loading huge dump fails RE: Re: Loading huge dump fails

Author scmadmin (OCN Member)
Full name Karthikeyan Sundharrajan
Date 2012-03-19 09:39:59 PDT
Message Thanks a lot, it worked ! Since Edge provides an option to dump in zip format, I thought it would work for Load as well. It appears, my assumption was wrong.
      So, dumped the repo in .dump format and loaded them successfully.

RE: Re: Loading huge dump fails RE: Re: Loading huge dump fails

Author scmadmin (OCN Member)
Full name Karthikeyan Sundharrajan
Date 2012-03-19 09:21:13 PDT
Message Hi Mark,
     Yes webserver process goes down ( Dump size is 8.5 GB - In zip format). I deleted the zip file and initiated the dump (.dump) format. Previously it was in zip format. Once the .dump is complete, I created a new project from the .dump backup and something seem to be happening. Will keep you posted.

PS : I am subscribed to this discussion forum. However, I am not receiving any e-mails.


Regards,
Karthik

> On Mar 19, 2012, at 10:56 AM, Karthikeyan Sundharrajan wrote:
>
> > I took a new dump (stored in the server - dump directory ) and attempted to create new repository from the backup as you suggested. The dump did appear in GUI. Unfortunately, when I click on Create button, it loads for a while and a message appears to check the progress in jobs.
> > When i click on Jobs, page cannot be found appears. So, I checked the csvn process in the server and it is already down.
> > Once the process goes down, I couldn't bring up the application because it times out. So, I have to delete the dump to bring up the csvn
>
> I do not understand what you are saying here. Are you saying the dump process is running but it brings down the web server process? And in order to bring the server back up you have to kill the running dump process?
>
> I do not have any explanation for that, but I guess you will need to run the svnadmin dump command from a command line on the server.
>
> Mark

Re: Loading huge dump fails Re: Loading huge dump fails

Author jeyanthan (OCN Member)
Full name Jeyanthan I
Date 2012-03-19 08:07:45 PDT
Message > Hi Mark,
> I took a new dump (stored in the server - dump directory ) and attempted to create new repository from the backup as you suggested. The dump did appear in GUI. Unfortunately, when I click on Create button, it loads for a while and a message appears to check the progress in jobs.
> When i click on Jobs, page cannot be found appears. So, I checked the csvn process in the server and it is already down.
> Once the process goes down, I couldn't bring up the application because it times out. So, I have to delete the dump to bring up the csvn


Consider checking console log to know what's happening at the back end.

--
Regards,
Jeyanthan

Re: Loading huge dump fails Re: Loading huge dump fails

Author markphip (OCN Member)
Full name Mark Phippard
Date 2012-03-19 08:06:02 PDT
Message On Mar 19, 2012, at 10:56 AM, Karthikeyan Sundharrajan wrote:

> I took a new dump (stored in the server - dump directory ) and attempted to create new repository from the backup as you suggested. The dump did appear in GUI. Unfortunately, when I click on Create button, it loads for a while and a message appears to check the progress in jobs.
> When i click on Jobs, page cannot be found appears. So, I checked the csvn process in the server and it is already down.
> Once the process goes down, I couldn't bring up the application because it times out. So, I have to delete the dump to bring up the csvn

I do not understand what you are saying here. Are you saying the dump process is running but it brings down the web server process? And in order to bring the server back up you have to kill the running dump process?

I do not have any explanation for that, but I guess you will need to run the svnadmin dump command from a command line on the server.

Mark

RE: Re: Loading huge dump fails RE: Re: Loading huge dump fails

Author scmadmin (OCN Member)
Full name Karthikeyan Sundharrajan
Date 2012-03-19 07:56:59 PDT
Message Hi Mark,
        I took a new dump (stored in the server - dump directory ) and attempted to create new repository from the backup as you suggested. The dump did appear in GUI. Unfortunately, when I click on Create button, it loads for a while and a message appears to check the progress in jobs.
        When i click on Jobs, page cannot be found appears. So, I checked the csvn process in the server and it is already down.
        Once the process goes down, I couldn't bring up the application because it times out. So, I have to delete the dump to bring up the csvn



Regards,
Karthik

Re: Loading huge dump fails Re: Loading huge dump fails

Author markphip (OCN Member)
Full name Mark Phippard
Date 2012-03-15 05:35:16 PDT
Message On Mar 15, 2012, at 5:49 AM, Dennis Wokke wrote:

> Did you succeed? I have the same problem, dump size is about 280MB.
> I tried Marks way but to no success.

What do you mean by no success? Are you saying you just do not see the dumps in the UI? You have to put them inside a folder in the dumps directory for them to show up on the create repository UI.

Mark

RE: Re: Loading huge dump fails RE: Re: Loading huge dump fails

Author dwokkel (OCN Member)
Full name Dennis Wokke
Date 2012-03-15 02:49:53 PDT
Message Hello Karthik,

Did you succeed? I have the same problem, dump size is about 280MB.
I tried Marks way but to no success.

Best regards,

Dennis

RE: Re: Loading huge dump fails RE: Re: Loading huge dump fails

Author scmadmin (OCN Member)
Full name Karthikeyan Sundharrajan
Date 2012-03-13 06:22:01 PDT
Message Hey mark ! I didn't notice that option while creating repository. I will try that option out.

Regards,
Karthik

Re: Loading huge dump fails Re: Loading huge dump fails

Author markphip (OCN Member)
Full name Mark Phippard
Date 2012-03-13 06:16:32 PDT
Message On Mar 13, 2012, at 9:12 AM, Karthikeyan Sundharrajan wrote:

> After mark helped me fix the dump issue yesterday, I was able to take the dump in zip format. The size of the dump is 8GB. I downloaded the dump to my local machine.

I would not try to upload a dump file that big via a browser.

> My suspect is that, http is not able to receive the file. In that case, is there a direct way to upload the dump file from UI ? If no GUI options, I can try to load it in traditional method.

When creating a new repository you should be able to pick from the backups available in your dumps folder. So you should not have needed to upload the file. You can just pick it when creating a repository and we will load using that file.

In your case that should have just worked, but if you had a dump file from elsewhere you can still use the feature by creating a folder in the dumps folder and then stick the dumpfile in it. I call my folder "dumpfiles" but you can name it anything. Once you do this, it will show up in the Create Repository UI.

Mark

RE: Loading huge dump fails RE: Loading huge dump fails

Author scmadmin (OCN Member)
Full name Karthikeyan Sundharrajan
Date 2012-03-13 06:14:23 PDT
Message I realized the difficulty in reading the log message. So,I attached it.
Attachments

Loading huge dump fails Loading huge dump fails

Author scmadmin (OCN Member)
Full name Karthikeyan Sundharrajan
Date 2012-03-13 06:12:27 PDT
Message Hi, After mark helped me fix the dump issue yesterday, I was able to take the dump in zip format. The size of the dump is 8GB. I downloaded the dump to my local machine. Created a new project and tried to upload the dump from local system and it fails. Below is the error i notice in console.log *************************************************** 2012-03-13 18:28:07.384:WARN::/csvn/repo/loadFileUpload org.codehaus.groovy.grails.web.pages.exceptions.GroovyPagesException: Error processing GroovyPageView: java.io.IOException: Closed at org.codehaus.groovy.grails.web.servlet.view.GroovyPageView.handleException(GroovyPageView.java:178) at org.codehaus.groovy.grails.web.servlet.view.GroovyPageView.renderWithTemplateEngine(GroovyPageView.java:151) at org.codehaus.groovy.grails.web.servlet.view.GroovyPageView.renderMergedOutputModel(GroovyPageView.java:79) at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:250) at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1063) at org.codehaus.groovy.grails.web.servlet.GrailsDispatcherServlet.doDispatch(GrailsDispatcherServlet.java:341) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1219) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:424) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:494) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:867) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:346) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115) at org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:292) at org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:260) at org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:251) at org.codehaus.groovy.grails.web.mapping.filter.UrlMappingsFilter.doFilterInternal(UrlMappingsFilter.java:183) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.obtainContent(GrailsPageFilter.java:245) at org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.doFilter(GrailsPageFilter.java:134) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:378) at org.springframework.security.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109) at org.springframework.security.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390) at org.springframework.security.ui.ExceptionTranslationFilter.doFilterHttp(ExceptionTranslationFilter.java:101) at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390) at org.springframework.security.providers.anonymous.AnonymousProcessingFilter.doFilterHttp(AnonymousProcessingFilter.java:105) at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390) at org.springframework.security.ui.AbstractProcessingFilter.doFilterHttp(AbstractProcessingFilter.java:277) at org.codehaus.groovy.grails.plugins.springsecurity.GrailsAuthenticationProcessingFilter.super$3$doFilterHttp(GrailsAuthenticationProcessingFilter.groovy) at sun.reflect.GeneratedMethodAccessor591.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:88) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058) at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070) at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:127) at org.codehaus.groovy.grails.plugins.springsecurity.GrailsAuthenticationProcessingFilter.doFilterHttp(GrailsAuthenticationProcessingFilter.groovy:56) at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390) at org.springframework.security.ui.logout.LogoutFilter.doFilterHttp(LogoutFilter.java:89) at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390) at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235) at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53) at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390) at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:175) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.codehaus.groovy.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:67) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.codehaus.groovy.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:65) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1190) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:424) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:457) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:867) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:245) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113) at org.eclipse.jetty.server.Server.handle(Server.java:337) at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:581) at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1020) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:775) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:228) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:474) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:437) at java.lang.Thread.run(Thread.java:619) 2012-03-13 18:28:07.386:WARN::HttpException(400,null,null) ********************************************* My suspect is that, http is not able to receive the file. In that case, is there a direct way to upload the dump file from UI ? If no GUI options, I can try to load it in traditional method.
Messages per page: